ATTORNEY DOCKET NO. 
063170.6600 



1 



PATENT APPLICATION 
09/721,806 



In The United States Patent and Trademark Office 
On Appeal From The Examiner To The Board 
of Patent Appeals and Interferences 



In re Application of: 

Serial No.: 

Filed: 

Art Unit.: 

Confirmation No.: 

Examiner: 

Title: 



Richard H. Harvey 

09/721,806 

November 24, 2000 

2164 

3613 

Sathyanaraya R. Pannala 

A METHOD AND APPARATUS FOR OPERATING A DATABASE 



Mail Stop Appeal Brief - Patents 

Commissioner for Patents 

P.O. Box 1450 

Alexandria, VA 22313-1450 



Dear Sir: 



Appeal Brief 

Appellant has appealed to the Board of Patent Appeals and Interferences ("Board") 
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respectfully submits this Appeal Brief. 
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Real Party In Interest 

The inventor assigned this application to Computer Associates Think, Inc., pursuant 
to an assignment recorded at reel 01 1925, frame 0775. Accordingly, the real party in interest 
is Computer Associates Think, Inc. 
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Related Appeals and Interferences 

Appellant, the undersigned Attorney for Appellant, and the Assignee know of no 
applications on appeal that may directly affect or be directly affected by or have a bearing on 
the Board's decision in the pending appeal. 
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Status of Claims 

Claims 1-5, 9, 10, 14-18, 22, 23, and 27-31 were finally rejected by the decision of the 
Examiner dated March 31, 2008. Claims 6-8, 1 1-13, 19-21, and 24-26 have been withdrawn. 
Appellant presents Claims 1-5, 9, 10, 14-18, 22, 23, and 27-31 for appeal and set forth these 
claims in Appendix A. 
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Status of Amendments 

All amendments submitted by Appellant were entered by the Examiner prior to the 
mailing of the Final Office Action dated March 31, 2008. 
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Summary of Claimed Subject Matter 

A database system may include an apparatus for processing a directory service query 
such as, for example, a X.500 or LDAP service query. (P. 1, 11. 4-8; p, 6, 11. 14-22). The 
system may represent the query as a logic expression. (P. 6, 11. 14-22; Fig. 2). The system 
may expand the logic expression into a sum of terms. Id. For example, a query that is 
expressed as A.(B+C.!D) may be expanded into the following sum of terms: A.B + A.C.ID. 
Expanding the expression into a sum of terms may permit the system to optimize the 
processing of the query. (P. 8, 11. 1-2). For example, expanding the expression into a sum of 
terms may provide an opportunity to flatten terms, to evaluate terms in any order, to evaluate 
terms in parallel, or to stop processing based on a size or time limit. (P. 8, 11. 1-7). 

To obtain the sum of terms, the system may remove nested instructions in order to 
simplify the expression representing the service query. (P. 8, 11. 15-17). Each term may then 
be converted into a flattened query. (P. 8, 11. 17-22). Thus, a relatively long expression can 
be converted into a number of smaller instructions (or expressions), each of which can be 
flattened. Id. The flattened and smaller instructions (or expressions) may run much faster 
than complex queries, thus improving performance of the system. Id, 

Once the expression is expanded into a sum of terms, the system may determine 
whether a particular term includes a NOT operator. (P. 6, 11. 23-27). If the term does not 
include a NOT operator, the system may convert the term to a SQL instruction. (P. 6, II. 26- 
27; Fig. 2a). However, if the term does include a NOT operator, then the term may be 
expanded into positive and/or negative terms. (P. 7, 11. 1-3), Thus, the query may be 
rewritten as an expression without NOT operators. (P. 9, 11. 15-26). For example, the term 
A.!B can be expressed as A.(l-B), which can be written as A - AB. Id. This expression no 
longer includes the NOT operator. Id. If the system supports subtraction, then the system 
may collect all positive terms in a list, collect all negative terms into another list, and then 
subtract the positive term list and the negative term list whilst ignoring duplicates, (P. 9, 1. 29 
-p. 10, 1. 2). If the system does not support subtraction, then the system may collect all 
negative terms in a list and, in the process of collecting all positive terms in another list, only 
keep the terms that are not in the negative list. (P. 10, 11. 3-5). 

In some embodiments, the system may process a service query with higher order 
subtractions. (P. 10, 11. 7-9). For example, the system may process a query of A.!B.!C into 
the following sum of terms: A - A.C - A.B +A.B.C. (P. 10, 11. 10-15). This expression can 
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be further processed (or evaluated) to remove or ignore duplicate or overlapping results. (P. 
10, 11, 16-25). The A operation may provide a positive list, the A.C operation may provide a 
negative list, and the A.B operation may also provide a negative list. Id. The lists may be 
evaluated in any order, or in parallel, or in accordance with the optimizations noted above. 
Id. By ignoring duplicate results, there may be no need to evaluate the term A. B.C. Id. With 
reference to mathematical principals, the subtraction of A.B and A.C may effectively subtract 
A. B.C. twice, which is a duplicate result. Id. The system described above may be applied to 
the evaluation / execution of database service queries, (P. 11, 11. 12-13). 

For the convenience of the Board, Appellant provides the following mapping of the 
independent claims here on appeal. Appellant does not necessarily identify all portions of the 
specification and drawings relevant to the recited elements of the claims. Appellant provides 
the following mapping to help the Board make a decision on this Appeal, not to limit the 
scope of the claims. 
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Claim 1 

A method of processing a database service query, comprising: {see, e.g., p. 6, 11. 14- 
22; p. 8, II. 26 - p. 9, L 2; Fig. 2) 

receiving a service query, {see, e.g,, p. 6, 11. 14-22; Fig. 2) 

obtaining a sum of terms associated with the service query by expanding at least one 
nested term into one or more un-nested terms, {see, e.g., p. 6, 11. 14-22; p. 7, 11. 24-29; p. 8, 11. 
1-24; p. 9, 1. 15 - p. 10, L 25; Figs. 2, 2a, 2b) 

evaluating the sum of terms as a plurality of SQL instructions, wherein the sum of 
terms comprises one or more positive terms and one or more negative terms, {see, e.g., p. 6, 
11. 23-30; p. 7, 11. 1-10; p. 8, 11. 8-29; Figs. 2, 2a, 2b) 

determining a plurality of results associated with the sum of terms, wherein the 
plurality of results are determined by a processor and the determination comprises: {see, e.g., 
p. 1, 11. 1-8; p. 6, 11. 14-22; p. 10, 11. 3-25; Figs. 2, 2a, 2b) 

collecting, into a first list, results associated with the one or more negative 

terms, and {see, e.g,, p. 10, 11. 3-6; p. 10, 11, 16-22) 

collecting, into a second list, results associated with the one or more positive 

terms while omitting from the second list any results that are in the first list, {see, e.g., 

p. 10, 11. 3-6; p. 10, 11. 16-22) 

and 

providing one or more results from the second list to a user, {see, e.g., p. 1, 11. 15-22) 
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Claim 9 

A system for processing a directory service query, comprising: (see, e.g., p. 1, 11. 1-8; 
p. 6, 11. 14-22; p. 8, 11. 26 - p. 9, 1. 2; Fig. 2) 

a database operable to store arbitrary data; and {see, e.g., p. 1, 11. 1-8; p. 8, 11. 14-24) 
a processor that is communicatively coupled to the database and that processes a 
service query by: {see, e.g., p. 6, 11. 14-22; p. 1, 11. 1-8) 

obtaining a sum of terms by expanding at least one nested term into one or 
more un-nested terms, {see, e.g., p. 6, 11. 14-22; p. 7, 11. 24-29; p. 8, 11, 1-24; p. 9, 1. 15 
-p. 10, 1. 25; Figs. 2, 2a, 2b) 

evaluating the sum of terms as a plurality of SQL instructions, wherein the 
sum of terms comprises one or more positive terms and one or more negative terms, 
{see, e.g., p. 6, 11. 23-30; p. 7, 11. 1-10; p. 8, 11. 8-29; Figs. 2, 2a, 2b) 

determining a plurality of results associated with the sum of terms, wherein 
the determination comprises: {see, e.g., p. 6, 11. 14-22; p. 10, 11. 3-25; Figs. 2, 2a, 2b) 

collecting, into a first list, results associated with the one or more 
negative terms, and {see, e.g., p. 10, 11. 3-6; p. 10, 11. 16-22) 

collecting, into a second list, results associated with the one or more 
positive terms while omitting from the second list any results that are in the 
first list, {see, e.g., p. 10, 11. 3-6; p. 10, 11. 16-22) 
and 

providing one or more results from the second list to a user, {see, e.g., p. 1, IL 

15-22) 
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Claim 14 

A method of processing a directory service query, comprising: {see, e.g., p. 6, 11. 14- 

22; p. 8, 11. 26 - p. 9, 1. 2; Fig. 2) 

receiving a directory service query, {see, e.g., p. 6, 11. 14-22; Fig. 2) 

obtaining a sum of terms by expanding at least one nested term into one or more un- 

nested terms, {see, e.g, p. 6, 11. 14-22; p. 7, 11. 24-29; p. 8, 11. 1-24; p. 9, 1. 15 - p. 10, 1. 25; 

Figs. 2, 2a, 2b) 

mapping the sum of terms to a plurality of SQL instructions, wherein the sum of terms 
comprises one or more positive terms and one or more negative terms, {see, e.g., p. 6, 11. 23- 
30; p. 7, 11. 1-10; p. 8, 11. 8-29; Figs. 2, 2a, 2b) 

determining a plurality of results associated with the sum of terms, wherein the 
plurality of results are determined by a processor and the determination comprises: {see, e.g., 
p. 1, II. 1-8; p. 6, 11. 14-22; p. 10, 11. 3-25; Figs. 2, 2a, 2b) 

collecting, into a first list, results associated with the one or more negative 

terms, and {see, e.g., p. 10, 11. 3-6; p. 10, 11. 16-22) 

collecting, into a second list, results associated with the one or more positive 

terms while omitting from the second list any results that are in the first list, {see, e.g., 

p. 10, 11. 3-6; p. 10, 11. 16-22) 

and 

providing one or more results from the second list to a user, {see, e.g., p. 1, 11. 15-22) 
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Claim 22 

A system for processing a directory service query, comprising: (see, e.g., p. 1, IL 1-8; 
p. 6, 11. 14-22; p. 8, 11. 26 - p. 9, 1, 2; Fig. 2) 

a database that is operable to store arbitrary data; and (see, e.g., p. 1, II. 1-8; p. 8, 11. 

14-24) 

a processor that is communicatively coupled to the database and that processes a 
directory service query by: (see, e.g., p. 6, 11, 14-22; p, 1, 11. 1-8) 

obtaining a sum of terms by expanding at least one nested term into one or 
more un-nested terms, (see, e.g., p. 6, 11. 14-22; p. 7, 11. 24-29; p. 8, 11. 1-24; p. 9, 1. 15 
-p. 10, 1. 25; Figs. 2, 2a, 2b) 

mapping the sum of terms to a plurality of SQL instructions, wherein the sum 
of terms comprises one or more positive terms and one or more negative terms, (see, 
e.g, p. 6, 11. 23-30; p. 7, 11. 1-10; p. 8, 11, 8-29; Figs. 2, 2a, 2b) 

determining a plurality of results associated with the sum of terms, wherein 
the determination comprises: (see, e.g., p. 6, IL 14-22; p. 10, 11. 3-25; Figs. 2, 2a, 2b) 

collecting, into a first list, results associated with the one or more 

negative terms, and (see, e.g., p. 10, 11. 3-6; p. 10, 11. 16-22) 

collecting, into a second list, results associated with the one or more 

positive terms while omitting from the second list any results that are in the 

first hst, (see, e.g, p. 10, 11. 3-6; p. 10, IL 16-22) 

and 

providing the determined plurality of results to a user, (see, e.g., p. 1, 11. 15-22) 
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Claim 31 

A method of processing a database service query, comprising: (see, e.g., p. 6, 11. 14- 
22; p. 8, 11. 26 - p. 9, L 2; Fig. 2) 

receiving a service query; {see, e.g., p. 6, 11. 14-22; Fig. 2) 

obtaining a sum of terms associated with the service query by: (see, e.g., p. 6, 11. 14- 
22; p. 7, 11. 24-29; Fig. 2) 

expanding at least one nested term into one or more un-nested terms; (see, 
e.g, p. 6, IL 14-22; p. 7, 11. 24-29; p. 8, 11. 1-24; p. 9, 1. 15 - p. 10, 1. 25; Figs. 2, 2a, 2b) 

expanding at least one term associated with at least one NOT operator into at 
least one negative term and at least one positive term; and (see, e,g., p. 7, 11. 1-10; p, 
9,1. 13 - p. 10,1. 6) 

if the service query comprises a term having at least two NOT operators, 
deleting from the sum of terms a third-order term corresponding to the term having at 
least two NOT operators; (see, e.g., p. 10, 1. 7 - p. 11,1. 1 1). 

evaluating the sum of terms as a plurality of SQL instructions; (see, e.g., p. 6, 11. 23- 
30; p. 7, 11. 1-10; p. 8, 11, 8-29; Figs, 2, 2a, 2b) 

obtaining a plurality of results wherein each term of the sum of terms is associated 
with one or more results; (see, e.g., p. 6, 11. 14-22; p. 10, 11, 3-25; Figs. 2, 2a, 2b) 

generating a first list comprising one or more results associated with the at least one 
negative term; (see, e.g., p. 10, 11. 3-6; p. 10, 11. 16-22) 

generating a second list comprising one or more results associated with the at least 
one positive term, wherein the first list and the second list are generated by a processor; (see, 
e.g, p. 10, 11, 3-6; p. 10, 11, 16-22; p. 6, 11. 23-30) 

removing or omitting from the second list one or more results associated with the at 
least one negative term; and (see, e.g., p. 10, 11. 3-6; p. 10, 11. 16-22; Fig. 2b) 

providing one or more results from the second list to a user, (see, e.g., p. 1, 11. 15-22) 
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Grounds of Rejection to be Reviewed on Appeal 

Appellant requests the Board to review: 

1) the rejection of Claims 1, 9, 14, 22, and 31 under the first paragraph of 35 
U.S.C. § 112; 

2) the rejection of Claims 1, 9, 14, 22, and 31 under the second paragraph of 35 
U.S.C. § 112; 

3) the rejection of Claims 1-5, 9-10, 14-18, 22-23, and 27-31 under 
35 U.S.C. § 101; and 

4) the rejection of Claims 1-5, 9-10, 14-18, 22-23, and 27-31 under 35 U.S.C. § 
103(a) as being unpatentable over U.S. Patent No. 6,356,892 Bl issued to 
Com, et al. {"Corn"), in view of U.S. Patent No. 6,1 12,198 issued to Lohman, 
et al. {"Lohman"), and in view of U.S. Patent No. 5,412,804 issued to Krishna 
( "Krishna "). 
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Argument 

For at least the following reasons, the Examiner's rejections of Claims 1-5, 9, 10, 14- 
18, 22, 23 and 27-31 are improper and should be reversed. 

I. Claims 1, 9, 14, 22, and 31 satisfy the requirements of 35 U.S.C. § 
112, first paragraph. 

The Office Action rejects Claims 1, 9, 14, 22, and 31 under Section 112, first 
paragraph, and alleges that these claims do not comply with the enablement requirement. In 
particular, the Office Action asserts that "determining a plurality of results... wherein the 
plurality of results are determined by a processor," as recited in Claim 1, is "not described in 
the specification in such a way as to enable one skilled in the art to which it pertains, or with 
which it is most nearly connected, to make and/or use the invention." (Office Action, p. 2). 
As shown below, this assertion is incorrect. 

"The test of enablement is whether one reasonably skilled in the art could make or use 
the invention from the disclosures in the patent coupled with the information known in the art 
without undue experimentation." M.P.E.P. § 2164.01 citing United States v. Teletronics, Inc., 
857 F.2d 778, 785 (Fed. Cir. 1988). There is no requirement that the specification provide 
concrete examples or illustrations of claimed steps. In fact, "[cjompliance with the 
enablement requirement of 35 U.S.C. § 112, first paragraph, does not turn on whether an 
example is disclosed." M.P.E.P. § 2164.02. All that is required is that the "information 
contained in the disclosure of an application must be sufficient to inform those skilled in the 
relevant art how to both make and use the claimed invention. Detailed procedures for making 
and using the invention may not be necessary if the description is sufficient to permit those 
skilled in the art to make and use the invention." M.P.E.P. § 2164. 

The Specification provides sufficient information and detail to enable those skilled in 
the art at the time of invention to make and use the claimed invention. The portion of Claim 
1 cited by the Examiner ~ "the plurality of results are determined by a processor" ~ is 
supported by the Specification. For example, the Specification describes an "apparatus for 
processing a directory service query" in order to collect "results" of the query. {See, e.g., p. 
6, 11. 14-22). In addition, the Specification provides several detailed examples regarding how 
to determine results for a directory service query. (See, e.g., p. 7, 1. 1 1-29; p. 10, 1. 7 — p. 1 1, 



DALOl: 1024768 



ATTORNEY DOCKET NO. 
063170.6600 



16 



PATENT APPLICATION 
09/721,806 



1. 11).^ Furthermore, the Specification describes "Relational Database Management 
Systems" as well as "systems used to provide directory services, such as X.500 service 
directories and LDAP service directories." {See, e.g., p. 1, 11. 4-8). Accordingly, the 
Specification is sufficient to inform those skilled in the relevant art hov^ to both make and use 
the claimed invention. Therefore, the rejection is improper and Appellant respectfully 
requests the Board to reverse the rejection of Claim 1 . 

In rejecting Claims 9, 14, 22, and 31 under Section 112, first paragraph, the Office 
Action employs the same rationale used to reject Claim 1. Accordingly, for reasons 
analogous to those stated above with respect to Claim 1 , Appellant respectfully requests the 
Board to reverse the rejections of Claims 9, 14, 22, and 31 . 

II- Claims 1, 9, 14, 22, and 31 satisfy the requirements of 35 U.S.C, § 
112, second paragraph. 

The Office Action improperly asserts that two elements of Claim 1 are indefinite. As 
shown below, these two elements are clear and adequately supported by the specification, 

A. "a first list„and..,a second list" 

The Office Action improperly asserts that the following portion of Claim 1 is 
indefinite: 

...the determination comprises: 

collecting, into a first list, results associated with the one or more 
negative terms, and 

collecting, into a second list, results associated with the one or more 
positive terms while omitting from the second list any results that are in the 
first list.... 

To support the rejection, the Examiner merely quotes the following portion of the 
Specification: 

It should be noted that a database that supports SQL may not supply a 
subtraction operator. In such instances a problem in processing the sum of 
terms as described above may arise. In order to process (evaluate) a 
subtraction, the method according to the present application: collects all 
positive terms in a list; collects all negative terms into another list; and then 



' In citing these portions of the Specification, Appellant does not intend to limit the claims to any particular 
embodiment. These portions of the Specification are merely cited to illustrate that Claim 1 complies with 35 
U.S.C. § 112. 
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subtracts the positive term list and the negative term Hst whilst ignoring 
duplicates. 

(Office Action, p. 3) (quoting Specification, p. 9, I. 27 -p. 10, 1. 2). The Office Action does 
not explain how or why the above portion of the Specification renders Claim 1 indefinite. 
Indeed, the cited elements of Claim 1 are clear and are sufficiently supported by the 
Specification. For example, another portion of the Specification, ignored by the Examiner, 
states: 

An alternative to the subtraction process noted above, is to collect all 
negative terms in a list, and in the process of collecting all positive terms in 
another list, only keep the terms that are not in the negative list. 

(Specification, p. 10, 11. 3-5).^ Thus, the Specification describes an alternative to subtraction. 
At least this portion of the Specification supports "collecting, into a first list, results 
associated with the one or more negative terms, and collecting, into a second list, results 
associated with the one or more positive terms while omitting from the second list any results 
that are in the first list" as recited in Claim 1 . Because Claim 1 is definite and adequately 
supported by the Specification, the rejection should be reversed. 

In rejecting Claims 9, 14, 22, and 31 under Section 1 12, second paragraph, the Office 
Action employs the same rationale used to reject Claim 1. Accordingly, for reasons 
analogous to those stated above with respect to Claim 1 , Appellant respectfully requests the 
Board to reverse the rejections of Claims 9, 14, 22, and 3 1 . 

B. ^Hhe plurality of results are determined by a processor'' 

The Office Action improperly asserts that another portion of Claim 1 — "the plurality 
of results are determined by a processor" — is indefinite. (Office Action, p. 3). The Office 
Action offers no explanation or support for this assertion. As shown below, the rejection is 
improper. 

The essential inquiry for determining compliance with 35 U.S.C. § 112, second 
paragraph, is "whether the claims set out and circumscribe a particular subject matter with a 
reasonable degree of clarity and particularity." M.P.E.P. § 2173.02. "The requirement to 
'distinctly' claim means that the claim must have a meaning discernable to one or ordinary 

^ In citing this portion of the Specification, Applicant does not intend to limit the claims to any particular 
embodiment. This citation merely shows that the disputed portion of Claim 1 is supported by at least a portion 
of the Specification. 
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skill in the art when construed according to correct principles . , . . Only when a claim remains 
insolubly ambiguous without a discemable meaning after all reasonable attempts at 
construction must a court declare it indefinite." Metabolite Labs. v. Lab. Corp. of Am. 
Holdings, 370 F.3d 1354, 1366, 71 U,S.P.Q.2d 1081, 1089 (Fed. Cir. 2004) (emphasis 
added). Furthermore, "a claim term that is not used... in the specification is not indefinite if 
the meaning of the claim term is discernable." Bancorp Services, L.L.C. v. Hartford Life Ins. 
Co., 359 F.3d 1367, 1372, 69 U.S.P.Q.2d 1996, 1999-2000 (Fed, Cir. 2004). 

The cited portion of Claim 1 — "the plurality of results are determined by a processor" 
— is clear. In addition, the claim language is supported by the Specification. The 
Specification provides several detailed examples regarding how to determine resuhs for a 
directory service query. {See, e.g., p. 7, 1. 1 1-29; p. 10, 1. 7 - p. 1 1, 1. 1 1), The Specification 
describes "Relational Database Management Systems" as well as "systems used to provide 
directory services, such as X.500 service directories and LDAP service directories," {See, 
e.g., p. 1, 11. 4-8). The Specification discloses an "apparatus for processing a directory 
service query" to collect "results" of the query. {See, e.g., p. 6, 11. 14-22). Thus, the cited 
portion of Claim 1 — "the plurality of results are determined by a processor" — is supported 
by the Specification and is clear to one of ordinary skill in the art. Accordingly, the rejection 
is improper and should be reversed. 

In rejecting Claims 9, 14, 22, and 31 under Section 1 12, second paragraph, the Office 
Action employs the same rationale used to reject Claim 1. Accordingly, for reasons 
analogous to those stated above with respect to Claim 1, Appellant respectfully requests the 
Board to reverse the rejections of Claims 9, 14, 22, and 31. 

III. Claims 1-5, 9-10, 14-18, 22-23, and 27-31 satisfy the requirements 
of 35 U.S.C. § 101. 

The Office Action rejects Claims 1-5, 9-10, 14-18, 22-23, and 27-31 under 35 U.S.C, 
§ 101 . (Office Action, p. 4). As shown below, this rejection is improper. 

A, Claims 1-5, 14-18, and 27-30 

The Office Action asserts that Claim 1 is merely directed to functional or 
nonfunctional descriptive material. (Office Action, p. 4). This assertion is incorrect. Claim 
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1 is directed to a "method of processing a database service query." Specifically, Claim 1 



A method of processing a database service query, comprising: 
receiving a service query, 

obtaining a sum of terms associated with the service query by expanding at 
least one nested term into one or more un-nested terms, 

evaluating the sum of terms as a plurality of SQL instructions, wherein the 
sum of terms comprises one or more positive terms and one or more negative terms, 

determining a plurality of results associated with the sum of terms, wherein 
the plurality of results are determined by a processor and the determination comprises: 

collecting, into a first list, results associated with the one or more negative 
terms, and 

collecting, into a second list, results associated with the one or more positive 
terms while omitting from the second list any results that are in the first list, 
and 

providing one or more results from the second list to a user. 

The method recited in Claim 1 is patentable. In the recent Comiskey decision, the 
Federal Circuit held that "[w]hen an unpatentable mental process is combined with a 
machine, the combination may produce patentable subject matter." In re Comiskey, 499 F.3d 
1365, 1379 (Fed. Cir. 2007). In Comiskey, the Examiner rejected all of the claims in an 
application directed to a system for mandatory arbitration. Id. at 1368. The Federal Circuit, 
however, observed that some of the claims (i.e., claims 17 and 46) recited the use of 
"modules" (e.g., a "registration module for enrolling", "an arbitration module for 
incorporating arbitration language", etc.). Id. at 1379. Accordingly, the Federal Circuit 
concluded that these "claims, under the broadest reasonable interpretation, could require the 
use of a computer." Id. Based on this conclusion, the Federal Circuit held that claims 17 and 
46, in "combining the use of machines with a mental process, claim patentable subject 
matter." Id. at 1380. Therefore, in Comiskey, the Federal Circuit reversed the rejection of 
claims 17 and 46 under 35 U.S.C. § 101. Id at 1381. 

This aspect of Comiskey is consistent with other Federal Circuit decisions which held 
that claims combining a mental process with a machine are patentable. See, e.g., State St, 
Bank & Trust Co. v. Signature Fin, Group, Inc., 149 F.3d 1368, 1371 (Fed. Cir. 1998) 
(holding patentable a "system that allows an administrator to monitor and record the financial 
information flow and make all calculations necessary for maintaining a partner fund financial 
services configuration" where a "computer or equivalent device [wa]s a virtual necessity to 
perform the task"). 



recites: 
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Like claims 17 and 46 in Comiskey, Claim 1 in the present application recites a 
machine or the use of a machine as part of the claimed invention. In particular, Claim 1 
recites that "the plurality of results are determined by a processor." Therefore, under the 
Federal Circuit's decision in Comiskey, Claim 1 is directed to patent eligible subject matter. 
Furthermore, the Examiner does not dispute that the method in Claim 1 provides a useful, 
concrete, and tangible result. Accordingly, Appellant respectfully requests the Board to 
reverse the rejection of Claim 1. 

In rejecting Claims 2-3, 5, 14-16, 18, and 27-30, the Examiner employs the same 
rationale used to reject Claim 1. Accordingly, for reasons analogous to those stated above 
With respect to Claim 1, Appellant respectfully requests the Board to reverse the rejection of 
Claims 2-3, 5, 14-16, 18, and 27-30. 

B. Claims 9-10, 22-23, and 31 

The Office Action rejects Claims 9-10, 22-23, and 31 under 35 U.S.C. § 101 and 

asserts that these claims "lack the necessary physical articles or objects to constitute a 

machine or manufacture." (Office Action, p. 4). This assertion is incorrect. Claim 9 is 

directed to a "system for processing a directory service query." Specifically, Claim 9 recites: 

A system for processing a directory service query, comprising: 
a database operable to store arbitrary data; and 

a processor that is communicatively coupled to the database and that processes 
a service query by: 

obtaining a sum of terms by expanding at least one nested term into 
one or more un-nested terms, 

evaluating the sum of terms as a plurality of SQL instructions, wherein 
the sum of terms comprises one or more positive terms and one or more 
negative terms, 

determining a plurality of results associated with the sum of terms, 
wherein the determination comprises: 

collecting, into a first list, results associated with the one or 
more negative terms, and 

collecting, into a second list, results associated with the one or 
more positive terms while omitting from the second list any results that 
are in the first list, 
and 

providing one or more results from the second list to a user. 

Thus, Claim 9 is directed a "system" that comprises "a database" and "a processor 
that... processes a service query." A "system" comprising a "database" and a "processor" is 

DALOl: 1024768 



ATTORNEY DOCKET NO, 
063170.6600 



21 



PATENT APPLICATION 
09/721,806 



clearly a machine or manufacture within the meaning of 35 U.S.C. § 101, The Examiner 
does not dispute that the system in Claim 9 provides a practical, useful, and concrete result. 
Accordingly, Claim 9 is directed to patent eligible subject matter. Therefore, Appellant 
respectfully requests the Board to reverse the rejection of Claim 9. 

In rejecting Claims 10, 22, 23, and 31 the Examiner employs the same rationale used 
to reject Claim 1. Accordingly, for reasons analogous to those stated above with respect to 
Claim 1, Appellant respectfully requests the Board to reverse the rejection of Claims 10, 22, 
23, and 31. 

IV. Claims 1-5, 9-10, 14-18, 22-23, and 27-31 are allowable over the 
cited references. 

The Office Action rejects Claims 1-5, 9-10, 14-18, 22-23, and 27-31 under 35 U.S.C. 
§ 103(a) as being unpatentable over U.S. Patent No. 6,356,892 Bl issued to Corn, et al. 
(''Corn"), in view of U.S. Patent No. 6,1 12,198 issued to Lohman, et al. (''Lohman"), and in 
view of U.S. Patent No. 5,412,804 issued to Krishna (''Krishna"), As shown below, the 
rejection is improper. 

A. Claims i, 3-5, 9-10, 14, 16-18, 22-23, 27-28, and 30 

The Office Action fails to support the rejection of Claim 1 for several reasons. First, 
the cited references fail to teach, suggest, or disclose that the "determination 
comprises... collecting, into a first list, results associated with the one or more negative terms, 
and collecting, into a second list, results associated with the one or more positive terms" as 
recited in Claim 1 . Second, the cited references fail to teach, suggest, or disclose "omitting 
from the second list any results that are in the first list" as recited in Claim 1. Third, the 
Corn-Lohman combination is improper because the proposed combination would render 
Corn unsatisfactory for its intended purpose. 

First, the cited references fail to teach, suggest, or disclose that the "determination 
comprises. ..collecting, into a first list, results associated with the one or more negative terms, 
and collecting, into a second list, results associated with the one or more positive terms" as 
recited in Claim 1 , In the Office Action, the Examiner relies on Corn for this aspect of Claim 
1. (Office Action, pp. 6-7). Corn teaches a method for re-writing LDAP queries as SQL 
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queries. (Corn; abstract). The cited portion of Corn describes merging sets of EIDs into an 

SQL query. (Col. 7, 11. 39-58). In particular, the cited portion of Corn states: 

As described above, according to the inventive method, for each LDAP 

filter element or sub-expression, there is a set of entries (EIDs) that will satisfy | 

the element. Thus, each element generally maps to a set of EIDs. The EID sets | 

are then merged together, preferably into a single SQL query, using a set of \ 

combination rules. Thus, if a pair of LDAP filter elements are subject to an [ 

LDAP logical OR operator, the corresponding EID sets are merged using an I 

SQL UNION logical operator. If a pair of LDAP filter elements are subject to i 

an LDAP logical AND operator, the corresponding EID sets are merged using I 

an SQL INTERSECT logical operator. If a pair of LDAP filter elements are I 

subject to an LDAP logical NOT operator, the corresponding EID sets are I 

merged using an SQL NOT IN logical operator. As will also be seen, these j 

combination rules are applied recursively such that all LDAP elements I 

associated with a particular logical operator are processed into the SQL query. \ 

This recursive processing facilitates handling of even complicated LDAP I 

queries having numerous layers of logical depth. | 

(Col, 7, 11. 39-58). Thus, Corn describes merging sets of EIDs "using a set of combination | 

rules." Id This portion of Corn, however, does not teach, suggest, or disclose "results 

associated with the one or more negative terms" or "results associated with the one or more 

positive terms" as recited in Claim 1. In addition, merely merging sets of EIDs, as described 

in Corn, does not teach, suggest, or disclose "collecting, into a first list , results associated 

with the one or more negative terms, and collecting, into a second list , results associated with 

the one or more positive terms" as recited in Claim 1 , (Emphases added.) Because the cited 

references fail to teach, suggest, or disclose this aspect of Claim 1, the cited references fail to 

support the rejection. 

Second, the cited references fail to teach, suggest, or disclose "omitting from the 

second list any results that are in the first list" as recited in Claim 1 . In the Office Action, the 

Examiner relies on Lohman for this aspect of Claim 1. (Office Action, p. 7). Lohman \ 

describes a method for parallel processing of subtasks associated with a query. (Abstract; I 

col. 1, 11. 42-59). The cited portion of Lohman describes duplicate elimination. (Col. 5, 11. 

30-33). Specifically, the cited portion states: 

Duplicate elimination (distinct) can be thought of as a special case of 
aggregation (with no aggregating functions); therefore, all of the parallelization 
decisions just described for aggregation apply to duplicate elimination as well. 

(Col. 5, 11. 30-34). Thus, the cited portion of Lohman includes a cursory reference to 
"duplicate elimination." Neither the cited portions of Corn or Lohman teach, suggest, or 
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disclose the "first list" or the ''second Ust" recited in Claim 1. Therefore, the cursory 

reference to "duplicate ehmination" in Lohman does not teach, suggest, or disclose "omitting 

from the second list any results that are in the first list" as recited in Claim 1 . Because the 

cited references fail to teach, suggest, or disclose this aspect of Claim 1 , the cited references 

fail to support the rejection. 

Third, the Corn-Lohman combination is improper because the proposed combination 

would render Corn unsatisfactory for its intended purpose. If a "proposed modification 

would render the prior invention being modified unsatisfactory for its intended purpose, then 

there is no suggestion or motivation to make the proposed modification." M.P.E.P. § 

2143.01. A "primary object" of Corn is to search a relational database to retrieve "target 

entries that exactly match given search criteria." {Corn; col. 2., IL 32-38). Corn describes a 

method for mapping an LDAP search query into an SQL query. (Corn; col. 2, 11. 51-54). To 

formulate the SQL query, the method in Corn must generate "unique entry identifier (EID) 

sets." (Corn; col. 3, 11. 1-20). "Each LDAP entry is assigned a unique identifier (EID)" and 

the EIDs are stored in an "Entry Table" and an "Attribute Table." (Corn; col. 5, 11. 59-65; 

coL 6, 11. 16-22). The "Entry Table" stores each EID with entry data, and the "Attribute 

Table" stores each EID with attribute values. (Corn; col. 5, 11. 58-67; coL 6, 11. 16-22). Corn 

formulates SQL queries by incorporating terms associated with EIDs. In particular, Corn 

requires that the EID sets be "merged together, preferably into a single SQL query." (Corn; 

col. 7, IL 42-44). For example. Corn presents the following SQL query: 

SELECT entry.EntryData, FROM LDAP_ENTRY as entry WHERE 
entry.EID in (SELECT distinct LDAP_ENTRY.EID FROM 
LDAP_ENTRY.ldap_desc WHERE (LDAP„ENTRY.EID=ldap_desc.DEID 
AND ldap_desc.AEID-<id>) AND LDAP_ENTRY.EID NOT IN ((SELECT 
EID FROM fl where fl=^ vL))). 

(Corn; col. 11, 11. 10-20) (emphases added). Notably, this SQL query comprises numerous 
terms associated with EIDs. Therefore, to retrieve data using this query, Corn must search 
the EIDs associated with LDAP entries. 

In contrast to Corn, the method in Lohman does not generate or store EIDs for each 
LDAP entry. Rather, Lohman describes a method for separating a query into "subtasks" and 
dividing a database into multiple "partitions." (Lohman; col. 2, 11. 53-60). Lohman then 
applies each subtask to an individual partition of the database. (Lohman; col. 2, IL 53-60). 
Combining Corn with Lohman would result in applying queries associated with EIDs to a 
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database without EIDs. A query or subquery for particular EIDs, as described in Corn, will 
simply not return results from a database that does not comprise any EIDs, as described in 
Lohman, Thus, the proposed combination would render Corn inoperable and therefore 
unsatisfactory for its intended purpose of retrieving "target entries that exactly match given 
search criteria." (Corn; col, 2., 11. 32-38). Because the proposed combination would render 
Corn unsatisfactory for its intended purpose, the combination is improper and should be 
withdravra. For at least the foregoing reasons. Appellant respectfully requests the Board to 
reverse the rejection of Claim 1 . 

In rejecting Claims 9, 14, and 22, the Examiner employs the same rationale used to 
reject Claim 1. Accordingly, for reasons analogous to those stated above with respect to 
Claim 1, Appellant respectfully requests the Examiner to reverse the rejections of Claims 9, 
14, and 22. 

Claims 3-5, 10, 16-18, 23, 27-28, and 30 depend from independent claims shown 
above to be allowable. In addition, these claims recite further elements that are not taught, 
suggested, or disclosed by the cited references. Accordingly, Appellant respectfully requests 
the Board to reverse the rejections of Claims 3-5, 10, 16-18, 23, 27-28, and 30. 

B. Claim 31 

The cited references fail to teach, suggest, or disclose each element of Claim 3 1 . For 
example, the cited references fail to teach, suggest, or disclose ''if the service query 
comprises a term having at least two NOT operators, deleting from the sum of terms a third- 
order term corresponding to the term having at least two NOT operators" as recited in Claim 
31. In the Office Action, the Examiner relies on Corn for this aspect of Claim 3 1 . (Office 
Action, p. 17). In particular, the Examiner cites a flowchart (Figure 6B of Corn) as well as 
the description in Corn that pertains to the flowchart. Corn explains that the flowchart 
illustrates a SQL generation algorithm. (Col. 7, 11. 62-64). The cited portion of Corn states: 

If the LDAP filter element includes neither AND nor OR, the routine 
continues at step 92 to determine whether the NOT logical operator is present. 
If so, the routine continues at step 94 to add the NOT IN logical operator to 
the SQL expression being generated. The routine then continues at step 96 to 
enter the recursive call so that all associated subexpressions may be parsed 
through the algorithm in the manner previously described. Thus, at step 98, a 
test is performed to determine whether all subexpressions associated with the 
NOT operator have been processed. If so, the routine returns at step 100; 
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otherwise, the routine loops back to step 96 and processes the next 
subexpression. 

(Col. 8, II. 40-50). Thus, the cited portion of Corn describes parsing subexpressions and, in 
some cases, adding the NOT IN logical operator to a SQL expression. This reference to 
parsing subexpressions, however, does not teach, suggest, or disclose "a term having at least 
two NOT operators" or "a third-order term corresponding to the term having at least two 
NOT operators" as recited in Claim 3 1 . Furthermore, there is nothing in the cited portion of 
Lohman that teaches, suggests, or discloses "if the service query comprises a term having at 
least two NOT operators, deleting from the sum o f terms a third-order term corresponding to 
the term having at least two NOT operators" as recited in Claim 31. (Emphasis added.) 
Because the cited references fail to teach, suggest, or disclose this aspect of Claim 31, the 
cited references fail to support the rejection. Accordingly, Appellant respectfully requests the 
Examiner to reverse the rejection of Claim 31. 

C Claims 2 and 15 

The cited references fail to teach, suggest, or disclose each element of Claim 2. For 

example, the cited references fail to teach, suggest, or disclose "expanding each term to 

remove NOT operators" as recited, in part, in Claim 2. The Examiner relies on Corn for this 

aspect of Claim 2. (Office Action, p. 7). Corn teaches a method for re-writing LDAP queries 

as SQL subqueries. (Corn; Abstract). The cited portion of Corn describes re-writing an 

LDAP logical operator NOT to an SQL logical operator NOT IN. (Corn-, col. 8, 11. 40-51). 

Specifically, the cited portion of Corn states: 

If a pair of LDAP filter elements are subject to an LDAP logical NOT 
operator, the corresponding EID sets are merged using an SQL NOT IN 
logical operator. 

(Corn; col. 7, 11. 50-52). Thus, Corn teaches re-writing a query term comprising a NOT 
operator as an SQL term comprising a NOT IN logical operator. Simply re-writing a query 
term to comprise a NOT IN operator does not teach, suggest, or disclose removing "NOT 
operators" as recited in Claim 2. Thus, the cited portion of Corn does not support the 
rejection of Claim 2. Accordingly, Appellant respectfully requests the Board to reverse the 
rejection of Claim 2. 
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In rejecting Claim 15, the Examiner employs the same rationale used to reject Claim 
2. Accordingly, for reasons analogous to those stated above with respect to Claim 2, 
Appellant respectfully requests the Board to reverse the rejection of Claim 15. 

/>. Claim 29 

The cited references fail to teach, suggest, or disclose each element of Claim 29. For 

example, the cited references fail to teach, suggest, or disclose "if the service query 

comprises a term having at least two NOT operators, deleting or disregarding from the sum of 

terms a third-order term corresponding to the term having at least two NOT operators" as 

recited in Claim 29. In the Office Action, the Examiner cites Corn with respect to this aspect 

of Claim 29. (Office Action, p. 16). The cited portion of Corn states: 

If the LDAP logical operator is NOT, the invention preferably excludes entries 
by negating the IN operation before the subquery. Thus, the combination rules 
includes, for example, mapping the LDAP logical OR operation to an SQL 
UNION, mapping the LDAP logical operation AND to SQL INTERCEPT, 
and mapping the LDAP logical operation NOT to SQL NOT IN, 

(Corn; col. 3, 11. 14-20). This portion of Corn merely teaches mapping the LDAP operator 
NOT to the SQL operator NOT IN. The cited portion of Corn makes no mention of "a third- 
order term" or of "a term having at least two NOT operators" as recited in Claim 29. In 
addition, the cited portion of Corn clearly fails to teach, suggest, or disclose "deleting or 
disregarding from the sum of terms a third-order term corresponding to the term having at 
least two NOT operators" as recited in Claim 29. Because the cited references fail to teach, 
suggest, or disclose the foregoing aspects of Claim 29, Appellant respectfully requests the 
Board to reverse the rejection of Claim 29. 
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Conclusion 



Appellant respectfully requests the Board of Patent Appeals and Interferences to 
reverse the rejection of Claims 1-5, 9, 10, 14-18, 22, 23 and 27-31 and instruct the Examiner 
to issue a notice of allowance as to all pending claims. 

The Commissioner is hereby authorized to charge the large entity fee of $510.00 
under 37 C.F.R. § 41.20(b)(2) for filing this Appeal Brief and the $120.00 one-month 
extension fee to Deposit Account No. 02-0384 of Baker Botts L.L.P. Although no other fees 
are believed to be due at this time, the Commissioner is hereby authorized to charge any 
additional fees and/or credit any overpayments to Deposit Account No. 02-0384 of Baker 
Botts L.L.P. 



Respectfully submitted. 



BAKER BOTTS L.L.P. 



Attorneys for Appellant 



Jusjnn N. Stewart 
Reg. No. 56,449 
(214) 953-6755 




Date: September 30, 2008 



CORRESPONDENCE ADDRESS: 



Customer No. 05073 
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Appendix A: Claims on Appeal 

1. (Previously Presented) A method of processing a database service query, 
comprising: 

receiving a service query, 

obtaining a sum of terms associated with the service query by expanding at least one 
nested term into one or more un-nested terms, 

evaluating the sum of terms as a plurality of SQL instructions, wherein the sum of 
terms comprises one or more positive terms and one or more negative terms, 

determining a plurality of results associated with the sum of terms, wherein the 
plurality of results are determined by a processor and the determination comprises: 

collecting, into a first list, results associated with the one or more negative 

terms, and 

collecting, into a second Hst, results associated with the one or more positive 
terms while omitting from the second list any results that are in the first Hst, 
and 

providing one or more results from the second list to a user. 

2. (Original) The method as claimed in claim 1, further comprising expanding 
each term to remove NOT operators. 

3. (Original) The method as claimed in claim 2, wherein the sum of terms are 
expanded using Boolean logic. 

4. (Original) The method as claimed in claim 1, in which the service query is an 
X.500 or LDAP service query. 

5. (Original) The method as claimed in claim 1, in which the service query is a 
search service query. 
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6. (Withdrawn) A method of processing a database service query, comprising: 
determining a SQL instruction representative of a function; 

listing the results of a subtracted SQL instruction in a first list, listing the results of a 
non-subtracted SQL instruction in a second list; and 

not listing results which are duplicates of previously listed subtracted or non- 
subtracted results. 

7. (Withdrawn) The method as claimed in claim 6, in which the service query is 
an X.500 or LDAP query. 

8. (Withdrawn) The method as claimed in claim 6, in which the service query is 
a search service query. 
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9. (Previously Presented) A system for processing a directory service query, 
comprising: 

a database operable to store arbitrary data; and 

a processor that is communicatively coupled to the database and that processes a 
service query by: 

obtaining a sum of terms by expanding at least one nested term into one or 
more un-nested terms, 

evaluating the sum of terms as a plurality of SQL instructions, wherein the 
sum of terms comprises one or more positive terms and one or more negative terms, 

determining a plurality of results associated with the sum of terms, wherein 
the determination comprises: 

collecting, into a first list, results associated with the one or more 

negative terms, and 

collecting, into a second list, results associated with the one or more 

positive terms while omitting from the second list any results that are in the 

first list, 

and 

providing one or more results from the second list to a user. 

10. (Previously Presented) The system as claimed in claim 9, further comprising 
means to perform X.500 or LDAP services. 
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1 1 . (Withdrawn) A directory service arrangement comprising: 

a database using a plurality of tables, each table having a plurality of rows and 
columns, and storing arbitrary data, and 

means for processing a service query by determining a SQL instruction representative 
of a function, listing the results of a subtracted SQL instruction in a first list, listing the 
results of a non- subtracted SQL instruction in a second list, and not listing results which are 
duplicates of previously listed subtracted or non-subtracted results. 

12. (Withdrawn) The directory service arrangement as claimed in claim 11, 
further comprising means to perform X.500 or LDAP services. 

13. (Withdrawn) A method for processing a database service query, comprising: 
translating a service query to an expression; 

simplifying the expression to a number of smaller expressions, each smaller expression being 
capable of being flattened; 

flattening each smaller expression; and 

executing each flattened expression. 



DALOl: 1024768 



ATTORNEY DOCKET NO. 
063170.6600 



32 



PATENT APPLICATION 
09/721,806 



14. (Previously Presented) A method of processing a directory service query, 
comprising: 

receiving a directory service query, 

obtaining a sum of terms by expanding at least one nested term into one or more un- 
nested terms, 

mapping the sum of terms to a plurality of SQL instructions, wherein the sum of terms 
comprises one or more positive terms and one or more negative terms, 

determining a plurality of results associated with the sum of terms, wherein the 
plurality of results are determined by a processor and the determination comprises: 

collecting, into a first list, results associated with the one or more negative 

terms, and 

collecting, into a second list, results associated with the one or more positive 
terms while omitting from the second list any results that are in the first list, 
and 

providing one or more results from the second list to a user. 

15. (Previously Presented) The method as claimed in claim 14, further 
comprising expanding each term to remove NOT operators. 

16. (Original) The method as claimed in claim 15, wherein the sum of terms are 
expanded using Boolean logic. 

17. (Previously Presented) The method as claimed in claim 14, in which the 
directory service query is an X.500 or LDAP service query. 

18. (Previously Presented) The method as claimed in claim 14, in which the 
directory service query is a search service query. 
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19. (Withdrawn) A method of processing a directory service query, comprising: 
determining a SQL instruction representative of the directory service query; 

listing the results of a subtracted SQL instruction in a first list, listing the results of a 
non-subtracted SQL instruction in a second list; and 

not listing results which are duplicates of previously listed subtracted or non- 
subtracted results. 

20. (Withdrawn) The method as claimed in claim 19, in which the service query 
is an X.500 or LDAP query. 

2 1 . (Withdrawn) The method as claimed in claim 1 9, in which the service query 
is a search service query. 
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22. (Previously Presented) A system for processing a directory service query, 
comprising: 

a database that is operable to store arbitrary data; and 

a processor that is communicatively coupled to the database and that processes a 
directory service query by: 

obtaining a sum of terms by expanding at least one nested term into one or 
more un-nested terms, 

mapping the sum of terms to a plurality of SQL instructions, wherein the sum 
of terms comprises one or more positive terms and one or more negative terms, 

determining a plurality of results associated with the sum of terms, wherein 
the determination comprises: 

collecting, into a first list, results associated with the one or more 
negative terms, and 

collecting, into a second list, results associated with the one or more 
positive terms while omitting from the second list any results that are in the 
first list, 
and 

providing the determined plurality of results to a user. 

23. (Previously Presented) The system as claimed in claim 22, further 
comprising means to perform X.500 or LDAP services. 



DALOl: 1 024768 



ATTORNEY DOCKET NO. 
063170.6600 



35 



PATENT APPLICATION 
09/721,806 



24. (Withdrawn) A directory service arrangement comprising: 

a database using a plurality of tables, each table having a plurality of rows and 
columns, and storing arbitrary data, and 

means for processing a directory service query by determining a SQL instruction 
representative of the directory service query, listing the results of a subtracted SQL 
instruction in a first list, listing the results of a non-subtracted SQL instruction in a second 
list, and not listing results which are duplicates of previously listed subtracted or non- 
subtracted results. 

25. (Withdrawn) The directory service arrangement as claimed in claim 24, 
further comprising means to perform X.500 or LDAP services. 

26. (Withdrawn) A method for processing a directory service query, comprising: 
translating a directory service query to an expression; 

simplifying the expression to a number of smaller expressions, each smaller 
expression being capable of being flattened; 

flattening each smaller expression; and 
executing each flattened expression. 
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27. (Previously Presented) The method of claim 1 wherein: 

evaluating the sum of temis comprises converting the sum of terms to a plurality of 
SQL instructions comprising at least one negative term; 
and further comprising: 

subtracting at least one result associated with the at least one negative term. 

28. (Previously Presented) The method of claim 1, wherein obtaining a sum of 
terms comprises: 

identifying at least one term associated with at least one NOT operator; and 
expanding the at least one term associated with the at least one NOT operator into at 
least one negative term. 

29. (Previously Presented) The method of claim 1, wherein: 

if the service query comprises a term having at least two NOT operators, deleting or 
disregarding from the sum of terms a third-order term corresponding to the term having at 
least two NOT operators. 

30. (Previously Presented) The method of claim 1, wherein obtaining a sum of 
terms comprises: 

identifying at least one term associated with at least one NOT operator; and 
expanding the at least one term associated with at least one NOT operator into at least 
one negative term and at least one positive term. 
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31. (Previously Presented) A method of processing a database service query, 
comprising: 

receiving a service query; 

obtaining a sum of terms associated with the service query by: 

expanding at least one nested term into one or more un-nested terms; 
expanding at least one term associated with at least one NOT operator into at 

least one negative term and at least one positive term; and 

if the service query comprises a term having at least two NOT operators, 

deleting from the sum of terms a third-order term corresponding to the term having at 

least two NOT operators; 

evaluating the sum of terms as a plurality of SQL instructions; 

obtaining a plurality of results wherein each term of the sum of terms is associated 
with one or more results; 

generating a first list comprising one or more results associated with the at least one 
negative term; 

generating a second list comprising one or more results associated with the at least 
one positive term, wherein the first list and the second list are generated by a processor; 

removing or omitting from the second list one or more results associated with the at 
least one negative term; and 

providing one or more results from the second list to a user. 



DALOl; 1024768 



ATTORNEY DOCKET NO. 
063170.6600 



38 



PATENT APPLICATION 
09/721,806 



Appendix B: Evidence 



U.S. Patent No. 6,356,892 issued to Corn, et al. 
U.S. Patent No. 6,112,198 issued to Lohman, et al. 
U.S. Patent No. 5,412,804 issued to Krishna, et al. 



Other than the above references, no evidence was submitted pursuant to 37 C.F.R. §§ 
1.130, 1.131, or 1.132, and no other evidence was entered by the Examiner and relied upon 
by Appellant in the Appeal. 



DALOl: 1024768 



United States Patent [i9] 

Lohman et al. 



[54] OPTIMIZATION OF DATA 

REPARTITIONING DURING PARALLEL 
QUERY OPTIMIZATION 

[75] Inventors: Guy Maring Lohman; Mir Hamid 

Pirahesh; Eugene Jon Shekita; David 
E. Simmen, all of San Jose; Monica 
Sachiye Urata, Saratoga, all of Calif. 

[73] Assignee: International Business Machines 
Corporation, Armonk, N.Y. 

[21] Appl. No,: 09/106,473 
[22] Filed: Jun. 29, 1998 

Related U.S. Application Data 

[60] Provisional application No. 60/051,259, Jun. 30, 1997. 



[51] Int. CI.'' G06F 17/30 

[52] U.S. CI 707/3; 707/3 

[58] Field of Search 707/2, 3, 4 

[56] References Cited 

U.S. PATENT DOCUMENTS 

5,745,746 4/1998 Jhingran et al 707,/2 

5,828,409 3/1999 Bam et al 707/2 

5,960,427 9/1999 Goel et al 707/4 

5,987,453 11/1999 Krishna et al 707/4 

6,009,265 12/1999 Huang et al 707/3 



illllllllllllllllllllllllllllllliillillllllllllllllll 

US006112198A 
[11] Patent Number: 6,112,198 
[45] Date of Patent: Aug. 29, 2000 



OTHER PUBLICATIONS 

Baru, C.K., "DB2 Parallel Edition," IBM Systems Journal, 
vol. 34 No. 2, pp. 292-322, 1995. 

Lohman, G., "Grammar-like Functional Rules for Repre- 
senting Query Optimization Alternatives," In Proceedings of 
the 1988 ACM SIGMOD Intn'l Conf. on Management of 
Data, pp. 18-27, 1988. 

Simmen, D., Fundamental Techniques for Order Optimiza- 
tion, In Proceedings of the 1996 ACM SIGMOD Intn 'I Conf 
on Management of Data, pp. 57-67, 1996. 
ISO and ANSI SQU Working Draft—btb. 5, 1993, Digital 
Equipment corporation, Maynard, Massachusetts. 

Primary Examiner — Paul V. Kulik 

Attorney, Agent, or Firm — Gray Cary Ware Freidenrich 

[57] ABSTRACT 

Query evaluation is optimized using parallel optimization 
techniques to make repartitioning more efiEcient. EfSciency 
is improved by recognizing the possible partitioning require- 
ments for achieving parallelism for a query operation, and 
by recognizing when the partitioning property of data sat- 
isfies the partitioning requirements of a query operation. A 
data base management system in accordance with the inven- 
tion uses parallel query processing techniques to optimize 
data repartitioning, or to avoid it altogether. 

44 Claims, 12 Drawing Sheets 



QEP 




table A is partitioned on A.y 
table B is partitioned on B..x 



Example QEP for a Directed Join 



U.S. Patent 



Aug. 29, 2000 



Sheet 1 of 12 



6,112,198 



table A 

X V z 

2 3 7 
1 5 8 

3 2 4 

4 1 2 



node 0 



table A 
(partition 0) 



2 



3 
1 



7 
2 



partitioning 
function 




node 1 



table A 
(partition 1) 



1 
3 



5 
2 



8 
4 



Figure 1; Example of Horizontal Partitioning 



U.S. Patent 



Aug. 29, 2000 



Sheet 2 of 12 



6,112,198 



node 0 



table A 
(partition 0) 

» ■ ■ X » • * 

2 
U 

table B 
(partition 0) 



2 



node 1 



table A 

(partition 1) 

« * ♦ X « ' ♦ 

1 
3 

table B 
(partition 1) 



1 
3 
1 



Figure 2: Example of Partitioned Join 
with no Data Movennent 



U.S. Patent 



Aug. 29, 2000 



Sheet 3 of 12 



6,112,198 



node 0 



node 1 



node 2 



table A 




(partition 


0) 


... X 


• • 


2 




U 




table B 




(partition 


0) 


... X 


* • 


1 
2 



table A 




(partition 


1) 


• • ♦ X • 




1 




3 




table B 




(partition 


1) 


... X 




3 
A 

1 



table B 
(partition 2) 



3 
1 
2 



Figure 3: Example of Partitioned Join 
that Requires Data Movement 



U.S. Patent 



Aug. 29, 2000 



Sheet 4 of 12 



6,112,198 



QEP 




table A is partitioned on A.y 
table B is partitioned on B..x 



Figure 4: Example QEP for a Directed Join 



U.S. Patent 



Aug. 29, 2000 



Sheet 5 of 12 



6,112,198 



Figure 5: Column Homogenization Process 

homogenize 
input: 

applied predicates PREDS, 
target tables TABLES, 
column to homogenize C 
output: 

result of homogenizing the column R 

1) set result R to NULL 

2) compute EQ the equivalence class for C using PREDS 

3) for (each column c_i of EQ while R is NULL) 
^) if (c_i belongs to a table in TABLES) then 

5) set result R to cJ\ 

6) endif 

7) endfor 

8) return R 
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1 2 

OPTIMIZATION OF DATA FIG. 3 is a representation of a data base management 

REPARTITIONING DURING PARALLEL system partitioned join operation ttiat requires data move- 

QUERY OPTIMIZATION ment 

FIG. 4 is a representation of a query execution plan (QEP) 

This Application claims the benefit of U.S. Provisional 5 for a directed join operation in a data base management 

Application Ser. No. 60/051,259, filed on Jun. 30, 1997. system. 

FIG. 5 is a representation of a column homogenization 
process in a data base management system. 

1. Field of the Invention FIG. 6 is a representation of a build-dir-req process using 
This invention relates generally to database management "naive" processing, for generating a requirement for a 

systems and, more particularly, to efficient evaluation of source stream from a target partitioning requirement, 

queries processed in relational database management sys- FIG. 7 is a representation of the build-dir-req process for 

tems. building a partitioning requirement for the source stream of 

2. Description of the Related Art 35 a join operation. 

A data base management system (DBMS) often uses FIG. 8 is a representation of a buiid-subq-req process that 

parallel query execution to deal with the performance is used in building a requirement for a local or directed 

demands imposed by applications executing complex que- subquery predicate application. 

ries against massive databases. Such parallelism is fre- FIG. 9 is a representation of a build-dir-req process in 

quently achieved by partitioning a database among proces- 20 accordance with the invention, which considers the effect of 

sors. The queries are broken into subtasks based upon the predicates binding partitioning key columns and considers 

partitioning of the database, so that different subtasks are correlation. 

assigned to different data partitions. The subtasks are FIG. 10 is a representation of an improved version of the 

executed by the processor managing the partition and the build-dir-req process in accordance with the invention, 

results of these subtasks are merged for delivery to an end 25 referred to as a listener-rp-pred process, 

user. Optimization choices regarding how queries are broken . . f 1 i . * 

f , , . i_ . , . . J T-u . FIG. 11 IS a representation of a local-agg-test process for 

into subtasks are driven by how the data IS partitioned. 1 hat 1, • • u u ™ 1^.^^ i^^^iu, 

. . * J . J / ■ u *u determining when aggregation can be completed locally, 

is, the partitioning property of the data determines how the & && & r 

queries are divided into subtasks. Often data has to be FIG. 12 is a representation of an improved version of the 

repartitioned dynamically to satisfy the partitioning require- 30 local-agg-test process illustrated in FIG. 11, performed in 

ments of a given query operation. Repartitioning is an accordance with the present invention, 

expensive operation and should be optimized or avoided DESCRIPTION OF THE PREFERRED 

altogether. EMBODIMENTS 

From the discussion above, it should be apparent that 

there is a need for a database management system that The present invention is implemented in a relational data 

evaluates complex query statements with reduced require- base management system (RDBMS) whose operations are 

ments for repartitioning of data and more efficient partition- represented in the drawing figures. It should be understood 

ing of the data. The present invention fulfills this need. that like reference numerals in the drawings refer to like 



SUMMARY OF THE INVENTION 



elements. 

ParalleHsm 



The present invention optimizes query evaluation by 
using parallel optimization techniques to optimize reparti- Parallelism is often used in conjunction with a computer 
tioning by recognizing: (1) the possible partitioning require- hardware architecture called shared-nothing to speed up the 
ments for achieving parallelism for a query operation, and execution of queries. In a shared-nothing architecture, a 
(2) when the partitioning property of the data satisfies the collection of processors (or nodes) execute queries in par- 
partitioning requirements of a query operation. A data base allel. A given query is broken up into subtasks, and all the 
management system in accordance with the invention uses subtasks are executed in parallel by the processors. Nodes in 
parallel query processing techniques to optimize data a shared-nothing architecture are typically connected by a 
repartitioning, or to avoid it altogether. These techniques high-speed communication network. The network is used to 
apply to join operations, aggregation operations, and opera- * coordinate subtasks across nodes and also to exchange data 
tions that apply subquery predicates. Unlike conventional between nodes. Each node has its own memory and disk, 
query processing techniques, these techniques exploit the To permit parallel query execution, tables are horizontally 
effect of data properties that arise from predicate application partitioned across nodes. Access to a given partition is only 
on the partitioning property of the data. through the node which manages that partition. The rows of 
Other features and advantages of the present invention a table are typically assigned to a node by applying some 
should be apparent from the following description of the deterministic partitioning function to a subset of the col- 
preferred embodiment, which illustrates, by way of umns. These columns are called the partitioning key of the 
example, the principles of the invention. table. A simple example illustrating how a table might be 

60 partitioned is shown in FIG. 1. 

In FIG. 1, Table A has been partitioned on a column A.x. 

FIG. 1 is a representation of horizontal partitioning of data Thus, the partitioning key is said to be A.x. The partitioning 

tables in a data base management system constructed in function assigns the rows of Table A to node 0 or to node 1. 

accordance with the present invention. In this example, rows with even values in A.x are assigned 
FIG. 2 is a representation of a partitioned join operation 65 to node 0, while rows with odd values in A.x are assigned to 

in a data base management system with no data movement node 1. Typically, the partitioning function is based on a 

required. simple hashing scheme. 
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Query Optimization in a Shared-Nothing RDBMS derived result with Table A in parallel. This join nriethod is 

The query optimizer in an RDBMS is responsible for commonly known as a directed join because it directs rows 

translating an SQL query into an efficient query execution ^ 1° Alternatively, the query optimizer can replicate the 

plan (QEP). The QEP dictates the methods and sequence ^^^^ ^^^^ Table A to all nodes of Table B. This jom method 

used for accessing tables. These methods are used to join ^ ^ broadcast join. It should be understood that the 

these tables, the placement of sorts, aggregation, predicate broadcast" will denote the replication of data across a 

application, and so on. The QEP is interpreted by the set of nodes. Finally, the optimizer could decide to reparti- 

database execution engine when the query is subsequently ^^^^^^ ^^"^^ completely different set of nodes, 

executed. There is added complexity in optimizing queries ^^^"^ the join column x to distribute rows. This 

for a shared-nothing architecture. Among other things, the ^« "^^'^^^^ "^^^^""^ ^VP^^^^^y ^^^^^^ ^ repartition join, 

added complexity involves determining how to break the ^^""| op^n^^zation, a QEP would be generated for each of 

QEP into subtasks and then merge the results of subtasks for f^^ ^j^^^^"^ ^^^^^^^^ mentioned above, assigned a cost 

delivery to an end user. based on an estimation of the QEP's execution time, and the 

_ . 1 .1 • u-. . .1 • least costly alternative would be chosen. The term "parti- 

In a shared-nothing architecture, the partitioning property ^ ^. j - - 7. -n u j * r * i , j- . j 

1 1^. 35 tioned loin will be understood to refer to local, directed, or 

IS used durine query optimization to keep track oi now a j • • . . • tt n i, j * ■ • »l 

, , . . j- N ,1 , ■• , repartitioned loin strategies. Unlike a broadcast lOin, these 

table (or intermediate) result has been partitioned across , , ■ fu.u.ui 

, ^ ^, .. . T -f i-rr strategies serve to loin partitions or both tables, 
nodes. The partitioning property describes two different 

aspects: (1) the nodes on which partitions of that table Partitioning Alternatives for Applying Subquery 
reside; and (2) the partitioning algonthm used to assign rows Predicates 
to nodes. Optimization choices about how queries are bro- 
ken into subtasks are driven by the partitioning property. As Parallel execution strategies for subquery predicate appli- 
an example, consider the query: cation are analogous to the parallel join strategies described 

above. As noted above, implementing parallehsm (that is, 
the parallelization decisions) for a join are hinged on how 

the two input tables are partitioned with respect to the join 

Q"^^y ^ predicate(s). Likewise, the parallelization decisions for 

jjciect * applying a subquery predicate hinge on how the table 

from A, B applying the subquery predicate and the table producing the 

where A.x = B.x subquery result are partitioned with respect to the subquery 

predicate(s). As an example, consider the query below: 

It should be noted that tables are referred to in queries by 
only their name, so that "A, B" is understood to refer to 
Table A and Table B. This convention will also be used 

occasionally in text. 35 

If Tables A and B are partitioned over the same nodes, use select * 

the same partitioning algorithm, and are both partitioned on ^ 

1 *i_ * • u c J • 11 1 -..u * froni A.x < > ALL (select B.x 

column X, then the join can be performed in parallel without ^^^^ g ^ 

any data movement. This is illustrated in FIG. 2, As shown, where B.y > o) 

no data movement is required to execute the join because the 49 

rows of A and B that satisfy the join predicate A.x=B.x are . . , r 1 1 • • -r ^ , , 1 

assigned to the same nodes. Parallel execution in this ^ '^e case for a local join, if Tables A and B are 

example is achieved by having a coordinator process. Partitioned over the same nodes, use the same parlitiomng 

replicate, and ship identical QEPs joining Table A and Table algorithm, and are both partitioned on column x then the 

B to node 0 and node 1. Each node performs its join 45 f^'i'^^'y P'^"^"""'^ ^^^^'^^ P"''"^' ^'"^o"' 

asynchronously and then returns rows back to the coordi- data movement. , , , , ■ . , 

^^^Qj. There are also directed, broadcast, and repartitioned ver- 
sions of plans for parallel subquery predicate application. 

Partitioning Alternatives for Joins For example, if the subquery table were partitioned on 

The join in the previous example gives rise to a partition- ,0 ^""^^^^f^ '\ ^""''^ ^''^^^^'^ ^^^^f /° 
ing requirement. The join requires that Table A and Table B " [^e nodes of A. Similarly, the subquery result could be 
are partitioned over the same nodes, use the same partition- broadcast to the nodes of A. It is also possible to defer the 
ing function, and are partitioned on column x. That is, their apphcation of the subquery predicate until after the table that 
partition property must be the same and they must be apphes the predicate has been repartitioned. Consequently, 
partitioned on the join column, x. If this was not the case, 55 o"'^ '^''^"^ °J broadcast the table applying the subquery 
then it would have been necessary to dynamically repartition '° . ^^ere the subquery result resides. Moreover, the opti- 
the tables to satisfy the partitioning requirement. '"'^er could decide to repartiUon both tables over some 
J . . . , completely different set of nodes prior to applying the 

Repartitioning is illustrated using the same query as in the , j- . t-l- ^ * ■ ; 

T A . /- ^.1 A • ■■ , subquery predicate. I his strategy is congruous to a reparti- 

previous example. As before, assume lable A is partitioned - ■ n-. . u j l j- < i- 

^ , ^ t ^ , T^.i-r.- -. 1 tioned loin. 1 ne term partitioned subquery predicate appli- 

on column x over nodes 0, 1 but now Table B is partitioned oo ^.jion" should be understood to refer to local, directed, or 

on some other column over nodes 0, 1, 2. This is shown m j u * r * . ■ 

, . , • -, 1 r . ■ , . ■ repartitioned subquery predicate apphcation strategies. 
FIG. 3, which provides an example oi a partitioned join 

operation that requires data movement. Partitioning Alternatives for Aggregation 

In the situation represented in FIG. 3, one or both of the 

tables needs to be dynamically repartitioned prior to execut- 65 The parallel execution strategy for aggregation is typi- 

ing the join. One option is to dynamically repartition Table cally carried out in two steps. Consider the following 

B using the join column x among nodes 0,1 and to join this example Query 3. 



Query 2 



6,112,198 

5 6 

a join predicate equaling each oF the corresponding columns 
of the partitioning key of A to some column of B. This 

criteria is not satisfied for A.x. A directed join strategy can 

Q"^^y ^ be constructed, however, by taking advantage of the fact that 

select count(*) ^ A.X is bound to a constant value. This information can be 

from A used to repartition B using the values "3" and B.y and then 

group by y a local join with A and the repartition result. 

Understanding of the rationale is best given via proof by 

Suppose that Table A is partitioned over multiple nodes contradiction. Suppose that B has been repartitioned using 

using the partitioning key x. In the first step of query ^0 the values 3 and B.y. Suppose that the join of A and the 

evaluation, aggregation is done on static partitions of A. repartitioned version of B cannot be done locally. Let k be 

Since A is not partitioned on the grouping column y, it is a value of B.y from a row on node n_i which joins with 

possible that rows in the same group reside on different some A row. Since the join cannot be done locally, there 

nodes. Thus, the result of the first step of aggregation results must exist some different node, n_j, where A.y=k. Observe 

in a partial count for each group on each node. In the final 15 that A.x=3 must be satisfied on aU nodes. Thus, both rows 

step, data is dynamically redistributed so that all rows with from A have the same values for the partitioning columns, 3 

the same y value are on the same node. Partial counts are and k. Since the partitioning algorithm is deterministic, n_i 

then added together to form the final result. and n_j must be the same node. 

The final aggregation step is unnecessary if data is ini- Failure to recognize that the join can be done locally 

tially partitioned such that all rows of the same group are on 20 would result in at least one of the tables being repartitioned 

one node. As described in the current literature, this is the or broadcast needlessly. Such a plan might be orders of 

case when the partitioning key columns are a subset of the magnitude worse in performance. The present invention 

grouping columns. It should be noted that the partitioning recognizes this. 

algorithm is deterministic. Two different rows will be Partitioning key columns can also be bound in a context- 
assigned to the same node if they have the same input 25 dependent way via correlated columns, as illustrated in the 
column values. Columns in the same group have the same following example ot Query 5: 
values for the grouping columns. Thus, if the preceding 

example is changed so that Table A is initially partitioned on 

y, the final aggregation step is unnecessary. 

Duplicate elimination (distinct) can be thought of as a 



Query 5 



special case of aggregation (with no aggregating functions); select * 

therefore, all of the parallelization decisions just described ^ 

1 . J T * V • 11 where B.Z > ANY (select count(*) 

tor aggregation apply to duplicate ehmination as well. from a 

where x = B.x; 

The Effect of Other Properties on Partition Analysis 35 

group by y 



50 



The term "partition analysis" will be used to designate the 

collection of processing involved in two tasks: (1) deter- Suppose that A is still partitioned as in the previous example. 

mining the partitioning requirements for a given operation; It was mentioned earlier that parallel aggregation requires 

and (2) when a partitioning property satisfies a given parti- two phases, unless the input to the initial aggregation phase 

tioning requirement. In the preceding sections, basic parti- is partitioned such that the partitioning key columns are a 

tion analysis for simple queries involving joins, subquery subset of the grouping columns. This condition does not 

predicate evaluation, and aggregation have been described. hold here, because Table A is partitioned using columns A.x 

To perform this task efficiently, the optimizer has to deal and A.y; nevertheless, the final aggregation step is not 

with the effect of other properties to avoid unnecessary necessary, because A.x is bound to the correlation value B.x. 

movement of data. The argument is similar to that provided for the previous 

As an example of how other properties come into play, example. The general effects of column equivalence and 

consider the following example Query 4: correlation must also be considered in partition analysis. 

This is described further below. 

The term "partition analysis" has been defined to refer lo 

algorithms for determining the partitioning requirements for 

QEP operations and for determining when a partitioning 

property satisfies these requirements. In particular, partition 

analysis has been described for join, subquery predicate 

evaluation, and aggregation queries. The effect of other 

properties on partition analysis also has been illustrated. 

- ui A ■ i-r 1 1.- 1 1 Specifically, the way in which application of predicates 

Suppose that Table A is partitioned over multiple nodes ^ ^\ . i • . . , -u ^ -i 

. .f. . ^ r 1 A J eo^ects partition analysis has been described. Failure to 

usme the composite partitioning key or columns A.x and . , , i • 

A o *u * ui D • f**- 1a ^.r^^tu^ ^ ^ ^^A^c- consider these effects can result in the unnecessary move- 

A.y. Suppose that Table B is partitioned over the same nodes ^ ^ , , . , „ , ^ 

usin column B 60 ^ ^ QEP? which may perform orders oi 

described above, the naive processing for determining magnitude worse than one that does consider these effects, 

the partitioning strategies for a join v/ould fail to determine ^ SUo^ms, and Properties 
that a jom strategy which directs tuples oi B to nodes oi A 

is a possibility. That is, they would fail to recognize that A query compiler for a relational data base management 
there is a way to repartition B on the nodes of A so that a 65 system (RDBMS) in accordance with the invention trans- 
local join could then take place between A and the result of lates a high-level query into a query execution plan (QEP). 
repartitioning B. These naive processes require that there is This QEP is then interpreted by a database engine at 



Query 4 


select * 






from A, B 






where A.x 


= 3 and A.y = B.y 


55 
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run-lime. Conceptually, a QEP can be viewed as a datallow One of Ihe relevanl properliesis Ihe tables properly, which 

graph of operators, where each node in the graph corre- keeps track of the relations that have been accessed and 

spends to a relational operation like join or a lower-level joined to form the tuples in the stream. Different accesses to 

operation like sort. Each operator consumes one or more the same table are recorded as separate entries in the tables 

input sets of tuples (i.e., relations), and produces an output 5 property. Another relevant property is the columns property, 

set of tuples (another relation). The input and output tuple The columns of a tuple stream can include both columns 

sets will be referred to as tuple streams. from base relations and those derived from expressions. The 

Each tuple stream includes a defining set of characteristics columns property is used to keep track of a tuple stream's 

or properties \cite{Lohman88. Examples of properties columns. Another property is the predicates property. During 

include the set of tables accessed and joined to form the 10 compilation, the predicates in a SQL query are typically 

tuples in the stream, the set of columns that make up each ^^^^^^ ^^^^ into conjunctive normal form. This allows 

tuple in the stream, the set of predicates that have been ^^"J^°f.^ f PP^^^ an independent predicate 

, ^ . r ,u When a predicate (actually conjunct) IS applied to an input 

applied to the tuples in the stream, the partitioning of the ^^^^^^^ each tuple in the resulting output stream has 

tuples in the stream, and so forth. ^^^^^^^^ ^^^^ ^^^^^^^ predicate. The predicates 

Each operator in a QEP determmes the properties of its 15 property is used to keep track of all the predicates that have 

output stream. The properties of an operator's output stream been applied to a tuple stream. 

are a function of its input stream(s) and the operation being Another relevant property relates to column equivalence, 

applied by the operator. For example, a sort operation por partition analysis, we are particularly interested in 

(SORT operator) passes on all the properties of its input predicates that equate columns, like EMPNO=DEPTNO. 

stream unchanged except for the order property. The repar- 20 These give rise to equivalence classes, which are sets of 

tition operation (RP operator) changes the partitioning prop- columns made equivalent by the application of equality 

erty of the tuple stream. A query compiler will typically predicates. Two columns can be made equivalent by a single 

build a QEP bottom-up, operator-by-operator computing predicate equating the two columns or by transitivity among 

properties as it goes. Note that, depending on the two or more equality predicates. For a given equivalence 

implementation, some properties may be stored in QEP 25 class, one column is arbitrarily chosen as the equivalence 

operators, while others may be recomputed when needed to class head. In the degenerate case, each column is equivalent 

save space. to itself and thus in some equivalence class. 

As a QEP is built, partitioning requirements for new For partition analysis, it is assumed that, given a set of 

operatorsmustbesatisfied, e.g., the input streams of the join applied predicates, there is some way to determine: (1) 

might need to be repartitioned or replicated so that they are 30 whether some column C_i is in the same equivalence class 

compatible for a parallel join. The join operation first as another column C_j; and (2) the equivalence class head 

determines a set of target partitionings, and then for each, for an arbitrary column C. It is important to point out that 

attempts to build a partitioning requirement for the join correlated columns are included in equivalence classes. For 

operands. If a join operand does not satisfy a partitioning example, if the conjunct X=A is appHed, where A is a 

requirement, an RP operator is added to the QEP for that 35 correlated reference to another query block, then X and Aare 

operand. FIG. 4 shows an example of a directed join assumed to be in the same equivalence class, 

operation. Also considered are predicates equating columns of tables 

The outer table. Table A, is not initially partitioned on the in an outer join operation in the same equivalence class tbr 

join column, x. The inner table, Table B, is partitioned on the partition analysis. For example, if A.X=B.X is a predicate in 

join column so the optimizer designates the inner table 40 an ON clause of an outer join operation between tables A and 

partitioning as the target partitioning. A partitioning require- B, A.X and B.X are considered to be in the same equivalence 

ment for the outer table is then derived from the target class. 

partitioning using the join predicates. The requirement con- Bound Columns 

sists of the nodes of the inner table and the join column. A -j -, • . r n • 

request is made for an outer table QEP that satisfies this 45 Bound columns also will be considered in the following 

partitioning requirement. Because none exists, an RP opera- description. As illustrated above, partition analysis also 

tor is added to an existing QER Partitioning requirements "^^^s to determine when a column has a constant value. A 

and the RP operator will be described in more detail below. column C is bound in a tuple stream if C has the same value 

, , . . . ^, * J * ■ for every tuple m the stream. Acolumn C can become bound 

With respect to join operations, other operators determine . • f 

partitioning requirements based upon analysis of the query, m a vanety o ways. 

and attempts to build a QEP which satisfies these require- (^) ^ can be derived from a constant expression; 

ments. For example, the aggregation operation (GROUP BY (b) there exists a predicate of the form C=10; 

operator) generates a partitioning requirement for local (c) a column in the equivalence class of C has become 

aggregation and tests the partitioning property of the input bound; or 

stream to determine if a final aggregation step is necessary. (d) columns which functionally determine C have become 

Relevant Properties Other than the Partitioning a 1 r- ^ u u^..^a ^« ^ ^^^t^^* 

A column C can also become bound m a context- 

roperty dependent way. For example, suppose we have C_„i=C_j 

Partition analysis is effected by more than the partitioning ^^d C_j is a correlated column. Then C_i is bound in the 

property. The relevant properties are briefly described 60 context where C_j is correlated. For partition analysis, it is 

below. The following description provides a general idea or assumed that, given a set of applied predicates, there is some 

references as to how relevant properties may be acquired or ^ay to determine if a column C in a tuple stream is bound 

changed by different operators may be accomplished. The g^d whether this property is context-dependent or not. 

details of such operations should be apparent to those skilled j 1 

in the art, in view of this disclosure. The description will 65 Correlated Columns 

focus on how each revSpective property is used in partition In contrast to the columns property, which essentially 

analysis. keeps track of columns which are flowed from base tables. 
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the correlated columns property keeps track of column 
values which are supplied by some other tuple stream. 

As an example, consider a join between two tables, Table 
T and Table R, using the join predicate T.C=R.C. Typically, 
an optimizer pushes down a join predicate so that it is 5 
applied when the inner table is accessed. Suppose that R is 
the inner table of a join and that the predicate has been 
pushed down to the access of R. Suppose further that R is 
accessed with an index scan ho as represented by an ISCAN 
operator. The ISCAN operator would apply TC=R.C and '^^ 
include TC in the correlated columns property to indicate 
that this value must be supplied by some other stream. When 
the tuple streams for R and T are finally joined, the join 
operator will remove T.C from the correlated columns 
property to indicate that the value is no longer needed as a 
correlation. 

In addition to correlated column references which occur 
due to the optimizer's decision to push down a join 
predicate, correlated column references can occur because a 
subquery references a column as an outer-reference. For 
partition analysis, it is assumed that, given a set of applied 
predicates, and a set of tables to which the predicates will be 
applied, there is some way to determine the correlated 
column references, lliis is essentially computed by subtract- 
ing the columns that can be supplied by the tables from those ''^ 
referenced in the predicates. 

The Partitioning Property 

The partitioning property of a tuple stream represents how 30 
the stream's tuples are distributed among nodes of the 
system. The partitioning property identifies: (1) the set of 
nodes that may contain tuples of the stream; and (2) the 
partitioning algorithm used for assigning tuples to nodes. 

A "nodegroup" is defined as a subset of the parallel 35 
system's nodes over which a tuple steam can be partitioned, 
riifferent streams may be distributed over the same node- 
group. Each nodegroup is assigned a unique identifier and 
this identifier is what is recorded in the partitioning property. 
Streams distributed over the same nodegroup, i.e. tables 40 
having the same nodegroup identifier, are said to be collo- 
cated. 

A partitioning algorithm is needed only for multi-node 
nodegroups. The partitioning algorithm is typically imple- 
mented with a deterministic function, like a hash function, 
applied to some subset of the columns of the tuple stream. 
These columns are called the partitioning key. Since the 
function is deterministic, tuples with the same values for the 
partitioning key are assigned to the same node. The parti- 
tioning key is represented by listing the columns to which 
the partitioning function is applied. For example, (C„l, 
C_2, C„3) is a representation of a partitioning key. It is 
assumed that the order of the columns in the partitioning key 
is relevant. Therefore, saying that the tuple stream is parti- 
tioned on (C„l, C_2) is different than saying that it is 
partitioned on (C„2, C_l). 

Specifically, the partitioning property is represented by 
three elements: 

(1) the nodegroup identifier; 

(2) the partitioning function identifier; and 

(3) the partitioning key. 

The latter two partitioning property elements are recorded 
only if the nodegroup contains multiple nodes. In certain 
cases, like in the case of a broadcast join, we may need to 65 
replicate tuples over all nodes in the nodegroup. The result- 
ing partitioning property is represented with the nodegroup 
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identifier and a special partitioning fimction which indicates 
that tuples are replicated. Thus, in this case, only the 
nodegroup identifier and partitioning function identifier are 
needed. 

Finally, two partitioning properties are said to be equiva- 
lent if they are collocated, use the same partitioning 
function, have the same number of partitioning columns in 
the partitioning key, and corresponding partitioning key 
columns are in the same equivalence class. Single node 
nodegroups are equivalent if they are collocated. Inherent in 
this definition is the assumption that the partitioning func- 
tion behaves the same regardless of the data types of its 
input. That is, if the predicate c_i=c_j evaluates to true, 
then it is assumed that the partitioning function produces the 

same value regardless of whether the value of c i or c_j is 

used. This simplifies the exposition without loss of gener- 
ality. 

Partitioning Requirements 

A partitioning requirement indicates a desired partitioning 
property for a QEP operation; consequently, a partitioning 
requirement has all of the same information included in a 
partitioning property. In general, it is assumed that a parti- 
tioning property and partitioning requirement are inter- 
changeable and that one can be derived from the other via a 
simple cast function (a function that simply changes the data 
type). It is said that a partitioning property satisfies a 
partitioning requirement if the two are equivalent after the 
partitioning requirement has been cast to a property. If no 
QEP satisfying a partitioning requirement exists, an RP 
operator is added to one or more existing QEP's. The 
partitioning requirement is passed as an argument to the RP 
operator. 

The RP Operator 

The RP operator is the only operator which can change the 
partitioning property of the data. One of its arguments is a 
partitioning requirement which must be satisfied. It achieves 
this by applying the partitioning algorithm specified by the 
partitioning requirement to each tuple of the input stream 
and sending the tuple to the node (or nodes in the case of a 
broadcast algorithm) which results. The nodes of the input 
stream to which the RP operator applies the partitioning 
algorithm are called the producer nodes, and the nodes 
which receive the tuples of the producer nodes are called the 
consumer nodes. It is assumed that all producer and con- 
sumer nodes are directly connected. 

When an RP operator is applied to a tuple stream con- 
taining correlated column references, data must flow both 
ways across connections. Correlated column values must 
first be sent from the consumer nodes to the producer nodes. 
The producer nodes then use the correlation values in their 
computations and send qualifying rows back to the con- 
sumer. This type of RP operator is called a listener RP. It is 
assumed that the node sending the correlation values, i.e. the 
node for which the producer is working on behalf of, is 
available on the producer side. The producer can use this 
node number to send rows back to the consumer. 
Alternatively, the consumer could supply its partitioning key 
columns and partitioning function to the producer. The 
producer could then apply the partitioning function to deter- 
mine the node that will receive data. The latter approach is 
not generally as efficient since it might require the producer 
to send columns which were not even referenced as corre- 
lations by the producer. In aca)rdance with the present 
invention, listener RP operations are minimized, since the 
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synchronization involved reduces parallelism. When a lis- Using the build-dir-req Process to Build a 

tener is unavoidable, the data flow is minimized by essen- Requirement for a Local or Directed Join 

tially pushing predicates which would be applied on the ^ ^ . , , ^ , , m i i • r 

% r»n * *u ^ A f Next, consider the use or the build-dir-req process tor 

consumer side or an RP operation onto the producer side 01 , .... ... . r , r 



the Operation, 



building a partitioning requirement for the source stream of 
a join. The build-join-req process represented in the pseudo- 
Column HomogenizatioD code of FIG. 7 illustrates the processing steps performed. 

, , . ^. 1 -^u • A * This process takes a QEP for one operand of a join, the 

The column homogenization algorithm is used to map , , ^ . r 7. . /.l /u • • 

, r ■ . / . J 1 • tables requirement for the source stream (the other join 

columns of requirements (such as order, key, or partitioning jx j • • j- < ,.^> . , 

, V , > . , V ^ in operand), and the torn predicates. The partitiomng property 

requirements) that are written in terras oi columns or one set -'^ ^ , ^' ^t-tV ■ j . . • • * 

, , • 1 . t c . r , LI of the target QEP provides the target partitionmg require- 

of tables to equivalent columns of another set of tables „ .i • • ^ t_ -u • • . 

^ . ro- T. 1 ^ -1.1 ment for the loin. The build-ioin-req process returns the 

\cite-^Simmen96. It maps a column to an equivalent column, . . . . , w u * u ^ u /^on 

, . ^ , , . • ^ ^ • 1 : 1 u 1 partitioning requirement which must be satisiied by a QhP 

subiect to the constraint that the equivalent column belongs ^„ . . ir * * * 

^ , p , . i_i T. . 1 • . 1 of the source stream. If the target partitioning requirement 

to one of a set of target tables. It takes as input a column, a , j ^ * ■ *l u m j j- 

, r , J,, , , , , , r ; IS can be mapped to the source stream via the build-du--req 

set of predicates, and the target tables. I he set or predicates . . .„ ^ . *u • u . 

.' , ^ - re, process, then it will return that; otherwise, it will return a 

defines the equivalence classes which are in efcect. . . . . i_ i_ i * * j r 

^ partitioning requirement which broadcasts to node oi the 

To illustrate the idea, suppose we have a partitioning target stream. Thus, the build-join-req process favors a local 

column A.X that we want to homogenize to the set of tables directed join over a broadcast join. The requirement 

{B, C} using predicate conjuncts A.X-B.X and A.X=C.X. returned by build-dir-req will trigger a local join if there is 

The equivalence class for A.X contains A.X, B.X, and C.X. " ^ ^^^^^^ Qj:p ^^^^ equivalent partitioning property and a 

There are two columns in the equivalence class which directed join if an RP operator must be added to an existing 

belong to one of the target tables; therefore, B.X or C.X is Q^p g^^-gfy partitioning requirement. 

returned arbitrarily. Changing the example slightly, suppose ^^^^.^^^ ^^^^ ^, 

we have the same column and predicates but the target tables ^ o r r ^ 

are {D, E}. In this case, there is no equivalent column which 

maps to the set of target tables so NULL is returned. 

FIG. 5 is pseudo-code that illustrates the processing Query 6 

performed by the column homogenization process. The ^ 

process searches the equivalence class and returns the first &omA B 

column belonging to the target tables. If no equivalent where A.x = B.x and A.y = B.y 

column satisfying the constraint exists, then NULL is — 

returned. If there are multiple possibilities for mapping, or 

homogenizing, the column, then one is returned arbitrarily. Suppose that A is partitioned over multiple nodes repre- 

Note that a column is equivalent to itself and thus can 3, rented by the nodegroup identifier 100, function identifier 

homogenize to itself if the table it belongs to is in the set of 100, and partitioning key (A.x, A.y). The build-jom-req 

target tables process is called with a QEP for A, the source table B, and 

the join predicates A.x=B.x and A.y=B.y. The build-dir-req 

A "Naive" Algorithm for Generating a Requirement process is able to return a requirement for a local or directed 

for a Partitioned Join or Subquery Predicate join by mapping the partitioning key columns to the source 

Application table. The resulting source requirement has a nodegroup 

A A • ■ t-r^»^A ,™^;^^f^ identifier of 100, a function identifier of 100, and a parti- 

Apartitioned join or partitioned subquery predicate appii- , . ^ ^ ' x rr- ^t-t^ r t. • i . 

cation requires lhat participating streants be equivalently t'°°!°.g ^ey of (B.x, B.y). If a QEP for B has an eqtjiva ent 

partitioned. There can be multiple such partitionings. One partitioning property, then_ the join can be done locally; 

i_ , . . * ' • „ + otherv/ise, an RP operator is added to an existing QbP, 

approach at coming up with a target partitioning requirement 45 "-"-"^^^^^^^s 1^ &^ 

for the operation is to cast the partitioning property of a QEP Note that it is not necessary to derive the target partition- 

of one of the streams to a requirement. The stream providing ing requirement from the partitioning property of an existing 

this partitioning is called the target stream. Any QEP^s QER One could start with any partitioning requirement as a 

whose partitioning property is broadcasting data are not target and then attempt to map it to both operands of the join 

permitted to provide the target partitioning. An equivalent 50 ^^e build-dir-req process. The build-join-req process 

partitioning requirement for the other stream is built, which ' shows just one apphcation of the build-dir-req process. The 

is called the source stream, by homogenizing each of the build-join-req process assumes that the transitive closure of 

partitioning key columns of the target partitioning require- the set of predicates has been completed; however, convcn- 

ment to columns of the source stream. tional naive algorithms say nothing about including corre- 

The build-dir-req process is illustrated in the pseudo-code 55 predicates which span the source and target tables, 

of FIG. 6 for generating a requirement for a source stream They also mention nothmg about join predicates which span 

from a target partitioning requirement. preserved and null-producing sides of an outer join 

The result partitioning is formed by using the nodegroup Moreover, they do not take advantage of predicates which 

, . . c. -J ^f*u bind columns to constant values or nor do they handle 

and partitioning function identifiers or the target partitioning . . 

requirement. The partitioning key of the result is formed by 60 correlation m a general way. 

homogenizing each of the columns of the target partitioning ^. b^iid-dir-req Process to Build a 

key to a column of one of the source tables using the Requirement for a Local or Directed Subquery 

equivalence classes defined by the input predicates. If any Predicate Application 

column cannot be homogenized to a column of a source 

table, then the NULL partitioning requirement is returned, 65 The build-dir-req process is also used for building a 

indicating that the join or subquery cannot be done in a requirement for a partitioned subquery predicate application, 

partitioned fashion. The build-subq-req process represented in FIG. 8 illustrates 
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its use in building a requirement for a local or directed 
siibquery predicate application. 

As in the build-join-req process, the partitioning property 
of the target QEP provides the target partitioning require- 
nient. The build-subq-req process returns the partitioning ^ 
requirement which must be satisfied by a QEP of the source 
stream. If the target partitioning requirement can be mapped 
to the source stream via build-dir-req, then it will return that; 
otherwise, it will return a partitioning requirement which 
broadcasts to the target stream. ^0 

The primary difference between the build-subq-req pro- 
cess and the build-join-req process is in the set of predicates 
supplied to build-dir-req for homogenizing the partitioning 
key. In the case of the join, build-join-req uses only con- 
juDCts applied by the join operand. When generating the 
subquery requirement, however, the build-subq-req process 
will use correlation predicates, and in certain cases, it can 
also use the subquery predicate (even when it is not a 
conjunct). 

Consider the following example of Query 7: 
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Like the naive algorithm, the improved build-dir-req 
process takes a target partitioning requirement, the tables of 
the source stream, and a set of predicates for mapping the 
partitioning key of the target partitioning requirement to 
columns of the source stream. The general idea is the same. 
The source partitioning is formed by using the nodegroup 
identifier and partitioning function identifier of the target 
partitioning. The partitioning key is formed by mapping the 
partitioning key column to the columns of the source stream. 

There are two key differences between the naive and 
improved versions of the algorithm: 

(1) A partitioning key column of the target partitioning 
requirement which is bound to a constant is considered 
mapped to the source stream by the constant; and 

(2) A partitioning key column of the target partitioning 
requirement which is referenced as a correlation by one 
of the source tables is considered mapped to the source 
stream by the correlation. 

The effect of each of these changes will be illustrated by 
considering again the example of Query 4, described above: 



Query 7 

25 

select * 
from A 

where P or A.y = ANY (select B.z 
from B 

where B.x = A.x)i 



Suppose that A provides the target partitioning for a 
parallel subquery predicate evaluation and that it is again 
partitioned over multiple nodes represented by nodegroup 
identifier 100 and function identifier 100 applied to parti- 
tioning key (A.x, A.y). 

The build-subq-req process is called with the correlation 
predicate B.x=A.x. It then adds the subquery predicate 

A. y=B.z and calls the build-dir-req process in an attempt to 
return a source partitioning requirement for a local or 40 
directed subquery predicate application. Using the correla- 
tion and subquery predicates, the build-dir-req process 
returns a source requirement with a nodegroup identifier of 
100, a function identifier of 100, and a partitioning key (B.x, 

B. z). If a QEP for B has an equivalent partitioning property 45 
then the join can be done locally; otherwise, an RP operator 

is added to an existing QEP. 

An Improved Process for Generating a 
Requirement for a Partitioned Join or Subquery 
Predicate Application 

When the build-dir-req process fails to determine that an 
equivalent partitioning requirement exisLs for the source 
stream, then either a broadcast join or broadcast subquery 
predicate application results. This is clearly less efficient 55 
than a local join or local subquery predicate application. It 
is also typically less efficient than one which directs tuples 
to the appropriate node. 

The build-dir-req process described in the previous sec- 
tion maps a target partitioning to a source stream given a set 60 
of predicates for mapping between the target and source 
streams. The algorithm generally mimics what is in the open 
literature, and as was illustrated earher, fails to consider the 
effect of predicates binding partitioning key columns. It also 
fails to handle correlation in a general way. The improved 65 
build-dir-req process shown in FIG. 9 remedies these short- 
comings. 



Query 4 

select * 
from A, B 

where A.x = 3 and A.y = B.y 



Suppose that Table A is partitioned over multiple nodes 
represented by nodegroup identifier 100, function identifier 
100, and partitioning key (A.x, A.y). Using the build-join- 
req process that calls the earlier version of build-dir-req will 
result in the return of a source requirement that broadcasts 
to nodegroup 100. The reason is that the earlier version of 
build-dir-req fails to homogenize partitioning key column 
A.x, since there is no join predicate mapping it to a column 
of the source table B; consequently, the NULL partitioning 
requirement is returned. 

The novel version of the build-dir-req process is able to 
generate a source partitioning requirement for a local or 
directed join. It is able to recognize that A.x is bound to the 
constant value 3 and maps it to the source stream using the 
constant. Using the join predicate to map the other parti- 
tioning key column, A.y, the source partitioning key for the 
join is (3, B.y). An RP operator which directs via partitioning 
key (3, B.y) can be added to an existing QEP for the source 
stream. 

Consider next the change to the build-dir-req process that 
deals with correlation. 



Query 8 

select * 
from A 

where P or A.y = ANY (select B.z) 
from B 

where B.x > A.x; 



Suppose that Table A provides the target partitioning for 
a parallel subquery predicate evaluation and that it is again 
partitioned over multiple nodes represented by nodegroup 
identifier 100 and function identifier 100 apphed to parti- 
tioning key (A.x, A.y). 

Using the build-subq-req algorithm which calls the earlier 
version of build-dir-req will result in the return of a source 
requirement which broadcasts to nodegroup 100. Again, the 
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reason is that the earlier version of build-dir-req fails to the subquery predicate A.y=B.z. Applying listener-rp-pred 

homogenize partitioning key column A.x since there is no to the tuple would reduce the likelihood of this happening, 

join predicate mapping it to a column of the source table B; It would apply the partitioning function to partitioning key 

consequently, the NULL partitioning requirement is values A.x-1 and B.z=2 and compare the result to the 

returned. The new version of build-subq-req is able to 5 caller's node. Assuming that tuples of table A having x=l 

generate a source partitioning requirement for a local or and y=2 and that tuples having x=l and y=l are assigned to 

directed subquery predicate application. It is able to recog- different nodes, this predicate would then avoid sending a 

nize that A.x is referenced as a correlation and maps it to the tuple which could not satisfy the subquery predicate. It is 

source stream as is. Using the subquery predicate to map the possible that another tuple on node N of Table A has and x=l 

other partitioning key column, A.y, the source partitioning lo and y=2. This is why the subquery predicate is still applied 

key for the subquery is (A.x, B.z). on the consumer side. This change offers some early filtering 

Thus, an RP operator which directs rows via partitioning in cases where this is not true. Note that if all of the columns 

key (A.x, B.z) can be added to an existing QEP producing in the result of build-dir-req are correlated, then the RP 

the subquery result. operator can avoid applying the listener-rp-pred process, 

15 since it will do no filtering. This explains why build-dir-req 

Changes to the Listener-RP Operator returns the NULL partition requirement when none of the 



partitioning key columns of the target map to source col- 
umns. 



Note that the RP operator added in the previous example 
would be a listener RP since the stream that the RP operator 
is applied to has a correlated column reference, A.x. Recall Using the Improved build-dir-req Process to Build 
thai a listener RP operator receives the correlation values " ^ Requirement for a Local or Directed Join 
from a consumer node, or caller, uses them in its . , , -.^ 
computations, and sends qualifying tuples back to the caller. Some changes are unplemented to the conventional build- 
It was indicated that the caller's node number is available to jom-req process. First of all, it will call the improved version 
the producer and that the RP operator simply uses it to ^ build-dir-req which exploits the bmding ot partitiomng 
determine where to send qualifying tuples as opposed to key columns to constants and their references as correla- 
having the caller pass its partitioning key values. A change tions. Consequently, in addition to join predicates, the pro- 
can be made to the behavior of the RP operator which will ^^ss will be called with local predicates and predicates 
take advantage of the improvements to build-dir-req. The referenced in the source stream which are correlated to the 
listener-rp-pred process represented in FIG. 1(» illustrates target stream. Outer join predicates are permitted as well, 
these changes. ^^^^^ Improved build-dir-req Process to Build 

FIG. 10 describes a predicate which is applied to a tuple ^ Requirement for a Local or Directed Subquery 

that the listener RP operator is getting ready to send back to Predicate Application 
the caller. If the RP operator has a target partitioning 

requirement with a partitioning key, it first applies the 35 A new version of the build-subq-req process is not 
partitioning function to these column values. If the node- implemented, but changes to the conventional process will 
number is the same as that of the caller, then the tuple is sent use the improved version of build-dir-req and that it will also 
back to the caller; otherwise, the tuple is not sent back. The supply local predicates as well as predicates referenced in 
effect of this change, in conjunction with the changes to the source stream which are correlated to the target stream, 
build-dir-req, is to apply (eitherjoin or subquery) predicates 40 We also note a change with regard to which subquery 
which will be applied on the consumer side of an RP predicates can supply a predicate to bmld-dir-req. A sub- 
operator on the producer side. q^iery predicate of the form column loALL column 2 is 
^ . , , . t r r> o • allowed to contribute to the mapping predicates and is 
Consider the previous example 01 Query 8 again. ,^ , , . n • . -r 

^ ^ / & passed to build-dir-req. This is semantically incorrect if one 

of the columns where to have a null value. It is noted that 
build-subq-req should exclude this predicate unless both 



Query 8 columns are not nullable. 



select * A Naive Algorithm for Determining If Aggregation 

from A Can be Completed Locally 

where P or A.y = ANY (select B.z) 50 

from ' It was noted that the parallel execution strategy for 

where B.x > A.x; aggregation begins by aggregating on each partition of the 

input stream. Then, if necessary, data is repartitioned so that 

Using the improved build-dir-req process, a source tuples in the same group land on the same node and a final 

requirement was determined for directing records from the 55 aggregation step takes place. It was noted that the final 

stream producing the subquery result to the stream applying aggregation step can be avoided if data is initially parti- 

the subquery predicate. The partitioning key of this require- tioned so that tuples in the same group are on same node, 

ment is (A.x, B.z). Using the predicate described above, the This is the case when the partitioning key columns of the 

listener RP operator could avoid sending many rows that input stream are a subset of the grouping columns, 

would not satisfy the subquery predicate A.y=B.z. eo The local-agg-test process shown in FIG. 11 is the cur- 

To illustrate this further, assume that a tuple on some node rently known test for determining when aggregation can be 

N of A has the values 1 for x and 1 for y and a tuple on a completed locally. 

different node of B has the values 10 for x and 2 for z. Note The process tests the partitioning property of the input 

that the B tuple satisfies the subquery predicate B.x>A.x. QEP against the grouping columns and returns true if either 

llierefore, if the listener RP operator did not apply the 65 (a) the QEP is on a single node; or (b) the QEP's partitioning 

predicate described by listener-rp-pred, it would send the key is a subset of the grouping columns. 1^e process returns 

record back to node N. However, this tuple will not satisfy false otherwise. 
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Note that the process takes column equivalence classes 
into account as shown in the following example of Query 9. 



Query 11 



——————— 5 select * 

Query 9 from B 



where B.z = (select count(*) 



select countC) f^om A 

froi" A v/here x = B.x and y = 3) 



where x = 
group by z,y 



Again, suppose that A is partitioned over multiple nodes 

using partitioning key (x, y). Note that x is bound to a 

Suppose that Table A is partitioned over multiple nodes correlation value and y is bound to a constant. Thus, each is 

using the partitioning key (x). After substituting the group- vacuously assumed to be contained in the set of grouping 

ing column, X, with the equivalent column, z, it is clear that is columns, even though there are none. Note that these 

the partitioning key columns are a subset of the grouping bindings effectively limit each execution of the subquery to 

columns. The effect of column equivalence classes is taken a single node. 

into account on line 11 where it considers a partitioning key . r *l t 

■r . • * t Advantages of the Invention 
column a member of the grouping columns ii it is in the 

same equivalence class as a grouping column. The algorithm This disclosure described novel techniques used in per- 

does not, however, take into account the effect of predicates forming partition analysis during query optimization for 

which bind partitioning key columns to constant or corre- joins, subquery predicate application, and aggregation. One 

lated values. Failure to lake these effects into account could of the main goals of partition analysis is to optimize or avoid 

result in unnecessary repartitioning of data. data repartitioning by recognizing the possible partitioning 

^■^ requirements for achieving parallelism for a query operation 

. , , „ f f • • Tf and when the partitioning property of the data satisfies the 

M Improved Process for Determmmg If partitioning requirements of a query operation. Unlike 

Aggregation Can be Completed Locally ^^^^^ techniques, the techniques presented here consider 

^ . , . r i_ 1 1 * , • the effect of predicates during partition analysis. 

The improved version of the local-agg-test process is 3Q ^ ^ , , r 

shown in FIG 12 ^ algorithm called build-dir-req was presented above tor 

generating a partitioning requirement for a partitioned join 

The improved version makes a simple yet powerful or partition subquery predicate application. Its use in deter- 

change to the previous version of the algorithm. Il assumes mining a requirement for a local or directed join was shown 

that any partitioning key column which is bound to a by the description of the build-join-req process. Also illus- 

constant or correlation value is vacuously a member of the trated was its use in determining a requirement for a local or 

grouping columns. In effect, we now say that aggregation directed subquery predicate application via the build-subq- 

can be completed locally if the unbound partitioning key req algorithm. 

columns are a subset of the grouping columns. The build-dir-req process is an improvement over the 

, 1-1. An current state of the art in that it takes into account predicates 

Consider the example of Query 10, where this change that bind partitioning key columns to constant values. It also 

prevents repartitioning. handles partitioning key columns which are referenced as 

correlations in a general way. The latter changes sometimes 

^ make it possible to filter tuples on the producer side of a 

Q^j^^y 45 listener RF operation which would not satisfy predicates 

- -— which will be applied on the consumer side. A listener-rp- 

select count(*) pj;ed process that takes advantage of this was described. 

wh^rtx = 3 Finally, a process called local-agg-test was described for 

group by y determining when aggregation can be completed locally. 

— Unlike known techniques, this process takes into account 

predicates which bind partitioning key columns to constant 
Suppose that Table A is partitioned over multiple nodes or correlated values, 
using partitioning key (x, y). It should be noted that the The present invention has been described above in terms 
partitioning key columns (x, y) are not a subset of the of a presently preferred embodiment so that an undersland- 
grouping columns (y). Aggregation can be completed 55 ing of the present invention can be conveyed. There are, 
locally, however, since partitioning key column, x, is bound however, many configurations for SQL-processing rela- 
te a constant value. The proof is best given by contradiction, lional data base management systems not specifically 
The argument is similar to that given above, for the example described herein but with which the present invention is 
that made use of constants to facilitate a directed join appHcable. The present invention should therefore not be 
operation limited to the particular embodiments described 

herein, but rather, it should be understood that the present 
Finally, note that if all of the partitioning key columns are invention has wide appHcability with respect to SQL- 
bound to a constant or correlation value, then the algorithm processing relational data base management systems gener- 
returns true regardless of what the grouping columns are. In ally. All modifications, variations, or equivalent arrange- 
fact, it will return true in this case if there are no grouping 65 ments and implementations that are within the scope of the 
columns as illustrated by the following example of Query attached claims should therefore be considered within the 
11, scope of the invention. 
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What is claimed is: 

1. A method of processing a query in a relational database 
management system that operates in a computer network to 
retrieve data from tables in computer storage, the method 
comprising the steps of: 5 

receiving two partitioned data streams that relate to a join 
operation; 

determining whether conjunct predicates can be used to 
locally perform a parallel inner join or outer join; and 

performing the join operation locally if the step of deter- lo 
mining indicates appropriate conjunct predicates can be 
used. 

2. A method as defined in claim 1, wherein the step of 
determining comprises applying conjunct predicates that 
equate columns of the same table. 

3. A method as defined in claim 1, wherein the step of 
determining comprises applying conjunct predicates that 
equate two columns, where one column is bound. 

4. A method as defined in claim 1, wherein the step of 
determining comprises applying conjunct predicates that 
equate columns of two different tables. 

5. A method as defined in claim 4, wherein neither one of 
the columns referenced by the column-equating predicates 
comprises a correlated column reference. 

6. A method as defined in claim 4, wherein one of the 
columns referenced by the column-equating predicates com- 2S 
prises a correlated column reference. 

7. A method as defined in claim 1, wherein the partitioned 
data streams comprise a source data stream and a target data 
stream, and the method further comprises the steps of: 

receiving a target query evaluation plan (QEP) that pro- 30 

duces tuples of the target data stream, which are to be 

joined with the source data stream; 
receiving a source tables requirement for the tables of the 

source data stream to be joined; 
receiving a set of source QEPs that produce tuples of the 

source data stream; 
receiving a set of conjunct predicates determined lo be 

suitable for locally performing a parallel inner join or 

outer join; 

generating a source partitioning requirement from a par- 
titioning property of the target QEF, the source tables 
requirement, and the set of conjunct predicates; and 

indicating that a local join is possible if there is a source 
QEP with a partitioning property that is equivalent to 
the source partitioning requirement. 

8. A method as defined in claim 7, wherein source 
partitioning requirements generated by the step of determin- 
ing are generated by performing a build-dir-req process, 
wherein the build-dir-req process comprises the steps of: 

setting a node group identifier of the source partitioning 
requirement to a node group identifier of the partition- 
ing property of the target QEP; 

setting a partitioning function identifier of the source 
partitioning requirement to that of a partitioning tunc- 
tion identifier of the partitioning property of the target 
QEP; 

forming the partitioning key of the source partitioning 
requirement by mapping partitioning key columns of 
the partitioning property of the target QEP to a column 50 
of the source data stream, wherein: 

a column of the partitioning key of the target QEP that is 
bound to a constant value C in the target data stream is 
mapped to the source data stream as C; 

a column of the partitioning key of the target QEP that is 65 
bound to a correlated value C in the target data stream 
is mapped to the source data stream as C; and 
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a column of the partitioning key of the target QEP that is 
in an equivalence class the same as a column C of the 
source data stream is mapped as C. 

9. A method of processing a query in a relational database 
management system that operates in a computer network to 
retrieve data from tables in computer storage, the method 
comprising the steps of; 

receiving two partitioned data streams that comprise a 

source data stream and a target data stream, wherein the 

data streams relate to a join operation; 
determining whether conjunct predicates can be used to 

direct tuples of the source data stream to the target data 

stream; and 

performing the join operation after directing the tuples of 
the source data stream to the target data stream if the 
step of determining indicates appropriate conjunct 
predicates can be used. 

10. A method as defined in claim 9, wherein the step of 
determining comprises applying conjunct predicates that 
equate columns of the same table. 

11. A method as defined in claim 9, wherein the step of 
determining comprises applying conjunct predicates that 
equate two columns, where one column is bound. 

12. A method as defined in claim 9, wherein the step of 
determining comprises applying conjunct predicates thai 
equate columns of two different tables. 

13. A method as defined in claim 12, wherein neither one 
of the columns referenced by the column-equating predi- 
cates comprises a correlated column reference. 

14. A method as defined in claim 12, wherein one of the 
columns referenced by the column-equating predicates com- 
prises a correlated column reference. 

15. A method as defined in claim 9, wherein the method 
further comprises the steps of: 

receiving a target query evaluation plan (QEP) that pro- 
duces tuples of the target data stream, which are to be 
joined with the source data stream; 

receiving a source tables requirement for the tables of the 
source data stream to be joined; 

receiving a set of conjunct predicates determined to be 
suitable for determining if the join can be done by 
directing tuples from nodes of the source data stream to 
nodes of the target data stream; 

generating a source partitioning requirement from a par- 
titioning property of the target QEP, the source tables 
requirement, and the set of conjunct predicates; and 

indicating that the join operation can be performed by 
directing tuples from nodes of the source data stream to 
nodes of the target data stream, if a source partitioning 
is generated, by returning the source partitioning 
requirement, otherwise indicating that the join opera- 
tion cannot be directed, by returning a broadcast par- 
titioning requirement. 

16. A method as defined in claim 15, wherein source 
partitioning requirements generated by the step of determin- 
ing are generated by performing a build-dir-req process, 
wherein the build-dir-req process comprises the steps of: 

setting a node group identifier of the source partitioning 
requirement to a node group identifier of the partition- 
ing property of the target QEP; 

setting a partitioning function identifier of the source 
partitioning requirement to that of a partitioning func- 
tion identifier of the partitioning property of the target 
QEP; 

forming the partitioning key of the source partitioning 
requirement by mapping partitioning key columns of 
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the partitioning property of the target QEP to a column 

of the source data stream, wherein: 

a column of the partitioning key of the target QEP that 
is bound to a constant value C in the target data 
stream is mapped to the source data stream as C; 5 

a column of the partitioning key of the target QEP that 
is bound to a correlated value C in the target data 
stream is mapped to the source data stream as C; and 

a column of the partitioning key of the target QEP that 
is in an equivalence class the same as a column C of lo 
the source data stream is mapped as C, 

17. A method of processing a query in a relational 
database management system that operates in a computer 
network to retrieve data from tables in computer storage, the 
method comprising the steps of: 15 

receiving two data streams that comprise a source data 
stream and a target data stream, w^herein a subquery 
predicate will be applied to the data streams; 

determining whether the subquery predicate can be 
applied to the target data stream locally in parallel; and 

applying the determined subquery predicate in parallel, 
locally, if the step of determining indicates the sub- 
query predicate can be so applied. 

18. A method as defined in claim 17, wherein the step of 
determining comprises applying conjunct predicates that 
equate columns of the same table. 

19. A method as defined in claim 17, wherein the step of 
determining comprises applying conjunct predicates that 
equate two columns, where one column is bound. 

20. A method as defined in claim 17, wherein the step of 
determining comprises applying conjunct predicates that 
equate columns of two different tables. 

21. A method as defined in claim 18, wherein neither one 
of the columns referenced by the column-equating predi- 
cates comprises a correlated column reference. 

22. A method as defined in claim 18, wherein one of the 
columns referenced by the column-equating predicates com- 
prises a correlated column reference. 

23. A method as defined in claim 15, wherein the step of 
determining comprises applying a subquery predicate that 
equates a column of a table in the target data stream with a 
column in the source data stream, and a subquery operator 
is either an ANY operator or an IN operator. 

24. A method as defined in claim 15, wherein the step of 
determining comprises applying a subquery predicate that 
equates a non-nullable column of a table in the target data 
stream with a non-nullable column in the source data stream, 
and a subquery operator is either a NOT ALL operator or a 
NOT IN operator. 

25. A method as defined in claim 15, wherein the parti- 
tioned data streams comprise a source data stream and a 
target data stream, and the method further comprises the 
steps of: 

receiving a target query evaluation plan (QEP) that pro- 
duces tuples of the target data stream, which are to be 
compared with tuples of the source data stream by a 
subquery predicate; 

receiving a source tables requirement for the tables of the 
data stream producing the subquery tuples; 

receiving a set of conjunct predicates determined to be 
suitable for deciding whether the subquery predicate 
can be applied to the target data stream locally, in 
parallel; 

generating a source partitioning requirement from a par- 65 
titioning property of the target QEP, the source tables 
requirement, and the set of conjunct predicates; and 



indicating that a local join is possible if there is a source 
QEP with a partitioning property that is equivalent to 
the source partitioning requirement. 

26. A method as defined in claim 25, wherein source 
partitioning requirements generated by the step of determin- 
ing are generated by performing a build-dir-rcq process, 
wherein the build-dir-req process comprises the steps of; 

setting a node group identifier of the source partitioning 
requirement to a node group identifier of the partition- 
ing property of the target QEP; 

setting a partitioning function identifier of the source 
partitioning requirement to that of a partitioning func- 
tion identifier of the partitioning property of the target 
QEP; 

forming the partitioning key of the source partitioning 
requirement by mapping partitioning key columns of 
the partitioning property of the target QEP to a column 
of the source data stream, wherein: 

a column of the partitioning key of the target QEP that is 
bound to a constant value C in the target data stream is 
mapped to the source data stream as C; 

a column of the partitioning key of the target QEP that is 
bound to a correlated value C in the target data stream 
is mapped to the source data stream as C; and 

a column of the partitioning key of the target QEP that is 
in an equivalence class the same as a column C of the 
source data stream is mapped as C. 

27. A method of processing a query in a relational 
database management system that operates in a computer 
network to retrieve data from tables in computer storage, the 
method comprising the steps of: 

receiving two partitioned data streams that comprise a 
source data stream and a target data stream, wherein a 
subquery predicate will be applied to the data streams; 

determining whether the subquery predicate and a con- 
junct predicate can be used to direct tuples of the source 
data stream to the target data stream; and 

directing the tuples of the source data stream to the target 
data stream if the step of determining indicates appro- 
priate predicates can be so used. 

28. A method as defined in claim 27, wherein the conjunct 
predicate in the step of determining comprises a conjunct 
predicate that equates columns of the same table. 

29. A method as defined in claim 27, wherein the conjunct 
predicate in the step of determining comprises a conjunct 
predicate that equates two columns, where one column is 
bound. 

30. A method as defined in claim 27, wherein the conjunct 
predicate in the step of determining comprises a conjunct 
predicate that equates columns of two different tables. 

31. A method as defined in claim 30, wherein neither one 
of the columns referenced by the column-equating predi- 
cates comprises a correlated column reference. 

32. A method as defined in claim 30, wherein one of the 
columns referenced by the column-equating predicates com- 
prises a correlated column reference. 

33. A method as defined in claim 27, wherein the step of 
determining comprises applying a subquery predicate that 
equates a column of a table in the target data stream with a 
column in the source data stream, and a subquery operator 
is either an ANY operator or an IN operator. 

34. A method as defined in claim 27, wherein the step of 
determining comprises applying a subquery predicate that 
equates a non-nullable column of a table in the target data 
stream with a non-nullable column in the source data stream, 
and a subquery operator is either a NOT ALL operator or a 
NOT IN operator. 
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35. A method as defined in claim 27, wherein Ihe method 
further comprises the steps of: 

receiving a target query evaluation plan (QEP) that pro- 
duces tuples of the target data stream, which are to be 
compared with tuples of the source data stream by a ^ 
subquery predicate; 

receiving a source tables requirement for the tables of the 
data stream producing the subquery tuples; 

receiving a set of conjunct predicates determined to be 
suitable for deciding whether the subquery predicate 
can be applied to the target data stream by directing 
tuples from nodes of the source data stream to nodes of 
the target data stream; 

generating a source partitioning requirement from a par- 35 
titioning property of the target QEP, the source tables 
requirement, and the set of conjunct predicates; and 

indicating that a local join is possible if there is a source 
QEP with a partitioning property that is equivalent to 
the source partitioning requirement, and otherwise indi- 20 
eating that the join operation cannot be directed and 
returning a broadcast partitioning requirement. 

36. A method as defined in claim 35, wherein source 
partitioning requirements generated by the step of determin- 
ing are generated by performing a build-dir-req process, 25 
wherein the build-dir-req process comprises the steps of: 

setting a node group identifier of the source partitioning 
requirement to a node group identifier of the partition- 
ing property of the target QEP; 

setting a partitioning function identifier of the source 
partitioning requirement to that of a partitioning func- 
tion identifier of the partitioning property of the target 
QEP; 

forming the partitioning key of the source partitioning 
requirement by mapping partitioning key columns of 
the partitioning property of the target QEP to a column 
of the source data stream, wherein: 
a column of the partitioning key of the target QEP that 
is bound to a constant value C in the target data 
stream is mapped to the source data stream as C; 
a column of the partitioning key of the target QEP that 
is bound to a correlated value C in the target data 
stream is mapped to the source data stream as C; and 
a column of the partitioning key of the target QEP that 
is in an equivalence class the same as a column C of 
the source data stream is mapped as C. 

37. A method of processing a query in a relational 
database management system that operates in a computer 
network to retrieve data from tables in computer storage, the 
method comprising the steps of: 

receiving a query that includes a relational operator that 
specifies one or more columns of tables in the computer 
storage that are distributed with a source partitioning 
across nodes of the computer network that are to be 55 
compared with one or more rows of tables in the 
computer storage that are distributed with a target 
partitioning across nodes of the computer network; 
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receiving a target partitioning requirement, tables of the 
source stream of data, and a set of predicates for 
mapping a partitioning key of the target partitioning 
requirement to columns of the source stream; 
performing a repartitioning correlated RP operator of the 
system in response to a request from a consumer node 
to deliver a repartitioned table to a target node, wherein 
the RP operator comprises the steps of: 
applying the source partitioning as a predicate opera- 
tion to a tuple that the RP operator is to send back to 
the consumer node; 
applying a partitioning function to column values if the 
RP operator is specified with the target partitioning 
requirement; 

sending the tuple back to the consumer node if the 
target node of the RP operator is the same as that of 
the consumer node; and otherwise not sending the 
tuple back to the consumer node, 

38. A method of processing a query in a relational 
database management system that operates in a computer 
network to retrieve data from tables in computer storage, the 
method comprising the steps of: 

receiving a partitioned data stream to which a table 
operation comprising either an aggregation operation 
or a distinct operation will be applied; 

determining whether conjunct predicates can be used to 
locally perform the table operation in parallel; and 

performing the parallel table operation locally if the step 
of determining indicates appropriate conjunct predi- 
cates can be used. 

39. A method as defined in claim 38, wherein the step of 
determining comprises applying conjunct predicates that 
equate columns of the same table. 

40. A method as defined in claim 38, wherein the step of 
determining comprises applying conjunct predicates that 
equate two columns, where one column is bound. 

41. A method as defined in claim 38, wherein the step of 
determining comprises applying conjunct predicates that 
equate columns of two different tables. 

42. A method as defined in claim 41, wherein neither one 
of the columns referenced by the column-equating predi- 
cates comprises a correlated column reference. 

43. A method as defined in claim 41, wherein one of the 
columns referenced by the column-equating predicates com- 
prises a correlated column reference. 

44. A method as defined in claim 38, further including the 
step of indicating that an aggregation operation can be 
performed locally, in parallel, if upon substituting columns 
of a key of the partitioning operation and columns of a key 
of the aggregating operation or of a key of the distinct 
operation with equivalence class heads, it is determined that 
unbound partitioning key columns are a subset of the 
aggregating or distinct key columns. 
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EFFICIENT IMPLEMENTATION OF complex queries lo get desired information from a backing 

LIGHTWEIGHT DIRECTORY ACCESS store. Increasingly, it has become desirable to use a rela- 

PROTOCOL (LDAP) SEARCH QUERIES tional database for storing LDAP directory data. Represen- 

WITH STRUCTURED QUERY LANGUAGE tative database implementations include DB/2, Oracle, 

(SQL) 5 Sybase, Informix and the like. As is well known. Structured 

Query Language (SQL) is the standard language used to 

This application includes subject matter protected by ^^^^^ ^^'^^ databases , 

copyright. All rights are reserved. SJ^^!^ for implementing an LDAP directory 

service with an relational database backing store (e.g., Dli/2) 

BACKGROUND OF THE INVENTION lo is to provide a design and implementation such that all 

LDAP search queries can be executed eHiciently with SQL. 

L Technical Field Implementing LDAP search queries with SQL, however, is 

This invention relates generally to providing directory a non^rivial task. On the one hand b^^^ 

'^^c ■ . A-^t^K..t^A ^™™,f,*r,r, «r,,,;r.^r,rr,^r.r S QL usc thc saffle AND , OR attd NOT logical operat ots, 006 

services in a distnbuted computing environment. , i • • i tt^at^ l 

possible solution to implementing LDAP search queries 
2. Descnption of the Related Art 15 ^^^^ might simply involve mapping each LDAP opera- 
A directory service is the central point where network lor to its corresponding SQL operator. This approach, 
services, security services and applications can form an however, does not work well in practice. Another approach, 
integrated distributed computing environment. Typical uses characterized by known prior art implementations (e.g., the 
of a directory services may be classified into several cat- Netscape b-tree LDAP server), involves retrieval of a super- 
cgories. A "naming service" (e.g., DNS and DCE Cell 20 qJ.- candidate entries, together with post-processing on 
Directory Service (CDS)) uses the directory as a source to those entries. There are several problems with this tech- 
locate an Internet host address or the location of a given nique. The two-step process is time consuming. More 
server. A "user registry" (e.g., Novell NDS) stores informa- problematic, however, is that negation and existence queries 
tion about users in a system composed of a number of give rise to a sequential search through the whole database, 
interconnected machines. The central repository of user 25 por a LDAP directory with a large number of entries, search 
information enables a system administrator to administer the results cannot be returned in an efiBcient manner, 
distributed system as a single system image. Still another The present invention addresses the problem of efficiently 
directory service is a "white pages" lookup provided by mapping an LDAP filter into an SQL query. 

uZ:^rf^'^-^Jr'''' 30 BRIEF SUMMARY OP THE INVENTION 

With more and more applications and system services It is a primary object of this invention to provide a method 

demanding a central information repository, the next gen- fof searching a relational database using hierarchical, filter- 

eration directory service will need to provide system admin- based queries, such as LDAP. 

istrators with a data repository that can significantly ease Another primary object is to provide an algorithm that 

administrative burdens. In addition, the future directory combines basic LDAP filter expressions into a preferably 

service must also provide end users with a rich information single SQL query that retrieves target entries that exactly 

data warehouse that allows them to access department or match given search criteria. 

company employee data, as well as resource information. Still another important object of this invention is to 

such as name and location of printers, copy machines, and provide a mechanism that can map even corapHcated LDAP 

other environment resources. In the Internet/intranet queries having infinite logical depth into SQL to facilitate a 

environment, it will be required to provide user access to relational database search. 

such information in a secure manner. Yet another important object of this invention is to map 

To this end, the Lightweight Directory Access Protocol LDAP logical operators efficiently for use in an LDAP 

(LDAP) has emerged as an IETF open standard to provide relational database search mechanism, 

directory services to applications ranging from e-mail sys- A more specific object of this invention is to efficiently 

terns to distributed system management tools. LDAP is an implement LDAP search queries with SQL wherein simple 

evolving protocol that is based on a client-server model in queries are combined together to form an arbitrary complex 

which a client makes a TCP/IP connection to an LDAP query that can retrieve target entries, preferably with no 

server, sends requests, and receives responses. The LDAP post-processing involved. 

information model in particular is based on an "entry," It is also an object of the present invention to provide a 

which contains information about some object. Entries are method for mapping LDAP search queries into an SQL 

typically organized in a specified tree structure, and each query that is efficient and does not degenerate into a sequen- 

entry is composed of attributes. tial search. 

LDAP provides a number of known functions including 55 It is another more specific object of the present invention 

query (search and compare), update, authentication and to provide a recursive algorithm that can deal with LDAP 

others. ^Ihc search and compare operations are used lo filler operators in a consistent way, and that deals with 

retrieve information from the database. For the search complicated LDAP queries with infinite number of logical 

function, the criteria of the search is specified in a search operators. 

filter. The search filter typically is a Boolean expression that 50 A more general object of this invention is to provide 
consists of qualifiers including attribute name, attribute hierarchical LDAP searches using relational tables in an 
value and Boolean operators like AND, OR and NOT, Users LDAP directory service having a relational database man- 
can use the filter to perform complex search operations. One agement system (DBMS) as a backing store, 
filter syntax is defined in RFC 2254. a more general object of this invention is to provide a 
LDAP thus provides the capability for directory informa- 65 reliable and scaleable enterprise directory solution, wherein 
tion to be efficiently queried or updated. It offers a rich set a preferred implementation is LDAP using a DB/2 backing 
of searching capabilities with which users can put together store. 
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The present invention solves the problem of ellicienlly is shown in FIG. 1. As is well-known, LDAP is the light- 
mapping an LDAP filter into an SQL query using unique weight directory access protocol, and this protocol has been 
entry identifier (EID) sets. According to the inventive implemented in the prior art, e.g., as either a front end to the 
method, a SQL subquery is first generated for each LDAP X.500 directory service, or as a standalone directory service, 
operator based on given translation rules. The SQL subquery 5 According to the protocol, a client machine 10 makes a 
generates a set of entry EIDs that match the LDAP basic TCP/IP connection to an LDAP server 12, sends requests 
operation. Thereafter, the SQL subqueries are combined into and receives responses. LDAP server 12 supports a directory 
a single SQL query according to a set of combination rules 21 as illustrated in a simplified form in FIG. 2. Each of the 
chosen corresponding to the logical operators of the LDAP client and server machines further include a directory "runt- 
filter query. Thus, for example, if the LDAP logical operator ime" component 25 for implementing the directory service 
is 0R(|), the invention then preferably uses an SQL UNION operations as will be described below. The directory 21 is 
to union the sets generated from the subquery. If the LDAP based on the concept of an "entry" 27, which contains 
logical operator is AND (&), the invention preferably uses information about some object (e.g., a person). Entries are 
an SQL INTERCEPT to intercept the sets generated from composed of attributes 29, which have a type and one or 
the subquery. If the LDAP logical operator is NOT, the more values. Each attribute 29 has a particular syntax that 
invention preferably excludes entries by negating the IN determines what kinds of values are allowed in the attribute 
operation before the subquery. ^Phus, the combination rules (^.g., ASCII characters, jpeg file, etc.) and how these values 
includes, for example, mapping the LDAP logical OR opera- constrained during a particular directory operation. 

tionto anSQLUmO^^^^ The directory tree is organized in a predetermined 

uon AND to SQL lOTEROT^^ the LDAP ^.^^ /^^^^ J ^^^^^ ^^^^^.^^ .^^ 

logical operation NOT to SQL NOT IN. 20 ^. ' . . « 1 a- *■ • u a » /orkxn 
^ \, . ^ ^, r 1 , siblmg entries by a "relative distinguished name (RDN). 
Generalizme, according to the preferred embodiment, a . ^^^r • *i * *• • u ^ -u^tl „.i„^ 
„ ^' ,. ^ ,5 1 J . u • An RDN comprises at least one distinguished attribute value 
method for searching a relational database using . , . _, . . ^ c u -u * • 
hierarchical, filter-based queries begins by parsing a filter- ^'^^ ^^^'^nM . ^^^^J ''f attribute is 
based query for elements and logical operators of the filter ^^^^ in the RDN. According to the protocol a globally 
query. For each filter element, the method generates an SQL 2^ unique name for an entry, referred to as a distmguished 
subquery according to a set of translation rules. For each name" (DN), comprises a concatenation of the RDN 
SQL subquery, the method then generates a set of entry sequence from a given entry to the tree root, 
identifiers for the filter query. Then, the SQL subqueries are The LDAP search can be applied to a single entry (a base 
combined into a single SQL query according to a set of level search), an entry's children (a one level search), or an 
combination rules chosen corresponding to the logical 30 entire subtree (a subtree search). Thus, the "scope" sup- 
operators of the filter query. ported by LDAP search are: base, one level and subtree. 

The foregoing has outlined some of the more pertinent LDAP does not support search for arbitrary tree levels and 

objects of the present invention. These objects should be V^^^ enumeration. 

construed to be merely illustrative of some of the more LDAP includes an application programming interface 

prominent features and applications of the invention. Many 35 (API), as described in "The C LDAP Application Program 

other beneficial results can be attained by applying the Interface", IETF Working Draft, Jul. 29, 1997, which is 

disclosed invention in a different manner or modifying the incorporated herein by reference. An application on a given 

invention as will be described. Accordingly, other objects client machine uses the LDAP API to effect a directory 

and a fuller understanding of the invention may be had by service "session" according to the flowchart of FIG. 3. At 

referring to the following Detailed Description of the pre- 40 step 40, an LDAP session with a default LDAP server is 

ferred embodiment. initialized. At step 42, an API function ldap_init( ) returns 

a handle to the client, and this handle may allow multiple 

BRIEF DESCRIPTION OF THE DRAWINGS connections to be open at one time. At step 44, the cUent 

For a more complete understanding of the present inven- authenticates to the LDAP server using, for example, an API 

tion and the advantages thereof, reference should be made to ldap_bind( ) function. At step 46, one or more LDAP 

the following Detailed Description taken in connection with operations are performed. For example, the API function 

the accompanying drawings in which: ldap_search( ) may be used to perform a given directory 

FIG, 1 is a representative LDAP directory service imple- search. At step 48, the LDAP server returns the results of the 

mentation; directory search, e.g., one or more database elements that 

FIG. 2 is a simplified LDAP directory; 50 meet the search criteria. The session is then closed at step 50 

FIG. 3 is a flowchart of an LDAP directory session; with the API ldap_unbind( ) function then being used to 

FIGS. 4A-AC show representative LDAP directory ser- close the connection, 

vice implementations having relational database backing H may be desirable to store LDAP directory data m a 

backing store. FIGS. 4A^C illustrates several representa- 

FIG. 5 is a simplified flowchart of the inventive method 55 live LDAP directory service implementations that use a 

for hierarchical LDAP searching in an LDAP directory relational database management system (RDBMS) for this 

service having a relational database management system as purpose. These systems merely illustrate possible LDAP 

a backing store using LDAP filter queries mapped efficiently directory services in which the present inverition may be 

to SQL' and implemented. One of ordinary skill should appreciate, 

FIGS, 6A-6D are a detailed composite flowchart of a 6° h°^ever, that the invention is not limited to an LDAP 

r , , . r • 4- ■ 1 -tu™ directory service provided with a DB/2 backing store. I he 

preferred embodiment of the inventive recursive algorithm ^^^^^^yy ov w ^ , . . 

r *u • *u * • J * iv.^ cr^T pnnciples or the present invention may be practiced in other 

of the invention that IS used to generate the SQL query. ^ ^ ^> - / v c-^an a • 

^ -1 ^ types of directory services (e.g., X.500) and using other 

DETAILED DESCRIPTION OF THE relational database management systems (e.g., Oracle, 

PREFERRED EMBODIMENT ^5 Sybase, Informix, and the like) as the backing store. 

A block diagram of a representative LDAP directory In FIG. 4A, an LDAP client 34 can connect to a number 

service in which the present invention may be implemented of networked databases 3Sa-58n through an LDAP server 
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36. The databases 38a-38« contain the directory informa- 
tion. However, frona the user's perspective, the LDAP server 
36 stores all the information without knowing the database 
38 in which the data is actually located. With this 
configuration, the LDAP server 36 is freed from managing 5 
the physical data storage and is able to retrieve information 
from multiple database servers 38 which work together to 
form a huge data storage. 

FIG. 4B illustrates a multiple client/multiple server 
LDAP/DB2 enterprise solution. In this environment, a DB/2 
client preferably runs on each LDAP server 36. Each such 
DB/2 client can connect to any database server 38 contain- 
ing directory information. The collection of database servers 
3Sa-3Sn form a single directory system image, and one or 
more of the LDAP servers 36 can access such information. 
Because all the LDAP servers 36 see the same directory '^^ 
image, a network dispatcher 37 may be deployed to route 
requests among the LDAP servers 36. 

FIG. 4C illustrates a multiple client/parallel super server 
configuration. In certain environments, where users need to 
store large amounts of information into the directory, this 
configuration automatically partitions the database into dif- 
ferent machines 38. In addition, database queries are divided 
into smaller, independent tasks that can execute 
concurrently, which increases end user query response time. 
This configuration enables users to store up to terabytes of 
data into the database. 

One of ordinary skill should appreciate that the system 
architectures illustrated in FIGS. 4A-4C are not to be taken 
as limiting the present invention. The inventive technique 
may be used to search any relational database using 
hierarchical, filter-based database queries. 

The technique is now described generally with reference 
to the flowchart of FIG. 5. This diagram illustrates the basic 
method of the invention to provide hierarchical LDAP 35 
searching in an LDAP directory service having a relational 
database management system (DBMS) as a backing store. 
The method begins at step 60 by parsing an LDAP filter- 
based query for elements and logical operators of the filter 
query. For each filter element, the method continues at step 
62 to generate an SQL subquery according to a set of 
translation rules, which will be defined below. For each SQL 
subquery, the method then continues at step 64 to generate 
a set of unique entry identifiers for the LDAP filter query. 
Then, at step 66, the SQL subqueries are combined into a 45 
single SQL query according to a set of combination rules 
chosen corresponding to the logical operators of the LDAP 
filter query. At step 68, the single SQL query is applied to the 
database step. Results are returned at step 70. 

The specific details of the routine illustrated in FIG. 5 are 50 
now described. By way of brief background, the inventive 
scheme preferably takes advantage of several LDAP table 
structures that are now described below. Further details 
about these structures are provided in U.S. Ser. No. 09/050, 
503 titled "A Fast And Efficient Method To Support Hier- 55 
archical LDAP Searches With Relational Tables", assigned 
to the assignee of this application, and incorporated herein 
by reference. 
Entry Table 

This table holds the information about a LDAP entry. This gQ 
table is used for obtaining the HID of the entry and sup- 
porting LDAP._SCOPE„.ONELEVEL and LDAP__ 
SCOPE . BASE search scope. 

EID. The unique identifier of the LDAP entry. This field 

is indexed. 65 
PEID. The unique identifier of a parent LDAP entry in the 
naming hierarchy. 
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EntryData. Entries are stored using a simple text format of 
the form "attribute: value". Non-ASCII values or val- 
ues that are too long to fit on a reasonable sized line are 
represented using a base 64 encoding. Giving an ID, the 
corresponding entry can be returned with a single 
SELECT statement. 
Descendant Table 

The purpose of this table is to support the subtree search 
feature of LDAP. For each LDAP entry with an unique ID 
(AEID), this table contains the descendant entries unique 
identifiers (DEID). The columns in this table are: 
AEID. The unique identifier of the ancestor LDAP entry. 

This entry is indexed. 
DEID. The unique identifier of the descend LDAP entry. 
This entry is indexed. 
Attribute Table 

One table per searchable attribute. Each LDAP entry is 
assigned an unique identifier (EID) by the backing store. The 
columns for this table are: 
EID 

Attribute value 

Thus, in the parent table, the EID field is the unique 
identifier of an entry in the LDAP naming hierarchy. The 
PEID field is the unique identifier of the parent entry in the 
naming hierarchy. In the descendant table, the AEID field is 
the unique identifier of a ancestor LDAP entry in the LDAP 
naming hierarchy. The DEID field is the unique identifier of 
the descend LDAP entry. 

In addition to the table structures described above, the 
following SQL SELECT statements are used by LDAP/DB2 
search routines; 
Base Level Search: 

SELECT entry.EntryData, 
from ldap„entry as entry 
where entry.EID in ( 

select distinct ldap„entry.EID 
from <lable list> 

where (ldap_entry.EID=<root dn id> ) 
<sql where expressions>) 
One Level Search: 
SELECT entry.EntryData, 
from ldap_entry as entry 
where distinct ldap_entry.EID 
from ldap_entry, <table list> 

ldap„entry as pchild, <list of tables> 
where ldap„entry.EID=pchild.EID 
AND pchild.PIED=<root dn id> 
<sql where expressions>) 
Subtree Search 

SELECT entry.EntryData, 
from ldap_entry as entry 
where entry.EID in ( 

select distinct ldap_entry.EID 

from ldap„entry, ldap_desc, <table list> 

where 

(LDAP_ENTRY.EID-ldap„desc.DElD AND 
ldap_desc,AEID=<root dn id>) 
ldap_entry as pchild. <table list> 
where ldap_entry.EID=ldap_desc.EID 
AND ldap_desc.AEID=%d <where expressions>). 
In the above representation, <table list> and <where expres- 
sion> are the two null terminated strings returned by the 
SQL generator. The <root dn id> is the unique identifier of 
the root dn. The where clause should only be generated if 
<where expression> is not the empty string and no errors 
where detected in the parsing the LDAP filter. 
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As is well-known, LDAP search queries coniprise six 
basic filters with the format <attribute> <operator> <value>. 
ConQplex search filters are generated by combining basic 
filters with Boolean operators AND (&), OR (|) and NOT (!). 
Translation Rules 5 

As described above in the flowchart of FIG. 5, an SQL 
subquery is generated for each LDAP filter element accord- 
ing to a set of translation rules. This was step 62 in the 
method. The following sets forth preferred translation rules 
used to generate SQL queries for basic LDAP filters accord- lO 
ing to the present invention: 
Equality 

LDAP filter 

(<attr> = <value>) 

^ ^ 15 

SQL Expression 

WHERE columnname = 'value' 

Ranges: 

LDAP filter 

(<altr> = <value>) 20 

(<attr> = <value>) 

SQL Expression 

WHERE columnname = 'value' 

WHERE columnname = 'value' ^5 
Substring: 
LDAP filter 

(<attr> = •<leading>'**-any"*-<trailing>") 
SQL Expression 
WHERE columnname 
LIKE '<leading>"* -any"* •<trailing>" 
Approximate: 
LDAP filter 

(<attr> = <value>) 35 
SQL Expression 

WHERE SOUNDEX (columnname) = SOUNDEX 
('value') 

As described above, according to the inventive method, 
for each LDAP filter element or sub-expression, there is a set 40 
of entries (EIDs) thai will satisfy the element. Thus, each 
element generally maps to a set of EIDs. The EID sets are 
then merged together, preferably into a single SQL query, 
using a set of combination rules. Thus, if a pair of LDAP 
fiher elements are subject to an LDAP logical OR operator, 45 
the corresponding EID sets are merged using an SQL 
UNION logical operator. If a pair of LDAP filter elements 
are subject to an LDAP logical AND operator, the corre- 
sponding EID sets are merged using an SQL INTERSECT 
logical operator. If a pair of LDAP filter elements are subject 50 
to an LDAP logical NOT operator, the corresponding EID 
sets are merged using an SQL NOT IN logical operator. As 
will also be seen, these combination rules are applied 
recursively such that all LDAP elements associated with a 
particular logical operator are processed into the SQL query. 55 
This recursive processing facilitates handling of even com- 
plicated LDAP queries having numerous layers of logical 
depth. 

With the basic translation rules and the EID sets approach, 
the present invention thus uses a recursive algorithm that 60 
handles complicated queries with a large number of layers of 
logical operators. The SQL generation algorithm of the 
present invention is illustrated in the composite flowchart of 
FIGS. 6A-6D. Note that the algorithm presented may be 
applied to search queries for all levels. 65 

With reference to FIG. 6A, the method begins at step 72 
by concatenating "(" to the generated SQL query (which, in 



the first iteration, is otherwise a skeleton query). The routine 
then tests at step 74 to determine whether the LDAP filter 
element includes the AND logical operator. If the outcome 
of the test at step 74 is positive, the AND logical operator is 
present. Typically, an AND logical operator sets apart at 
least a pair of subexpressions, and thus the routine includes 
appropriate logic to handle each subexpression separately in 
a recursive manner. To this end, the routine continues at step 
76 and sets an process variable nextFilter equal to 
ProcessFilter(DextFilter). At step 78, the routine concat- 
enates the SQL INTERSECT operator to the SQL query. 
This step maps the LDAP AND logical operator in the 
manner previously described. At step 80, a test is made to 
determine whether nextFilter is nil, i.e. whether all subex- 
pressions associated with the AND logical operator in the 
LDAP filter have been processed. If not, the routine returns 
at step 82. If, however, the outcome of the test at step 80 is 
negative, the routine loops back to step 76 to process the 
next subexpression associated with the AND logical opera- 
tor currently being mapped. 

If the outcome of the test at step 74 is negative, which 
indicates that the AND logical element is not in the filter 
element, the routine branches to step 84 to determine 
whether the LDAP OR logical operator is present. If the 
outcome of the test at step 84 is negative, the routine 
branches to FIG. 6B. If, however, the outcome of the test at 
step 84 is positive, the routine continues at step 86 with a 
recursive call into the ProcessFilter(nextFilter) subroutine 
previously described. Just as with the AND logical operator, 
an OR logical operator will typically include subexpressions 
that must be processed in a recursive manner. 

With reference now to FIG, 6B, the routine then continues 
at step 88 to concatenate the SQL UNION operator into the 
SQL query. At step 90, a test is performed to determine 
whether there are any more subexpressions to test. If not, the 
routine returns at step 92. If, however, the outcome of the 
test at step 90 is negative, the routine loops back to step 86 
to process the next subexpression associated with the OR 
logical operator currently being mapped. 

If the LDAP filter element includes neither AND nor OR, 
the routine continues at step 92 to determine whether the 
NOT logical operator is present. If so, the routine continues 
at step 94 to add the NOT IN logical operator to the SQL 
expression being generated. The routine then continues at 
step 96 to enter the recursive call so that all associated 
subexpressions may be parsed through the algorithm in the 
manner previously described. Thus, at step 98, a test is 
performed to determine whether ail subexpressions associ- 
ated with the NOT operator have been processed. If so, the 
routine returns at step 100; otherwise, the routine loops back 
to step 96 and processes the next subexpression. 

The remainder of the flowchart describes the mapping of 
simple filter elements using the set of translation rules 
previously described. At step 102, a test is made to deter- 
mine whether the LDAP filter includes a simple equality 
statement (e.g., "a=l"). If so, the routine continues at step 
104 to concatenate the associated attribute column name 
value into the SQL query. Thereafter, or if the result of the 
test at step 102 is negative, the routine continues in FIG. 6C. 

At stop 106, a test is made to determine whether the 
LDAP filter element is a substring filter. If so, the routine 
continues at step 108 to concatenate the associated attribute 
column name value into the SQL query. Thereafter, or if the 
result of the test at step 106 is negative, the routine continues 
at step 110 to determine whether the LDAP filler element is 
a greater than or equal expression. If so, the routine contin- 
ues at step 112 to concatenate the associated attribute 
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column name substring value into the SQL query being 
constructed. Thereafter, the routine continues at step 114 to 
deternaine whether the LDAP filter elenient is a less than or 
equal to expression. If so, the routine concatenates the 
associated value into the SQL expression at step 115 and 
continues at step 116. Step 116 is reach also as a result of a 
negative outconae of step 114. 

At step 116, a test is performed to determine whether the 
LDAP filter expression is a simple exists filter. If so, the 
routine concatenates the associated value into the SQL 
expression at step 118. The routine then continues with step 
120 in FIG. 6D, which is also reached by a negative outcome 
to the test at step 116. Step 120 tests whether the LDAP filter 
includes the simple approximate filter. If so, the appropriate 
value is concatenated into the SQL query at step 122. At step 
124, the SQL query is closed by concatenating an ")" value 
to complete the processing. 

The following is a detailed listing of a preferred code 
implementation of the above-described recursive algorithm: 
BEGIN PROCESSHLTER ALGORITHM ON AN LDAP- 
FILTER. 

CONCATENATE X TO SQL-EXPRESSION. 
DO ONE OF THE FOLLOWING DEPENDING ON 

THE TYPE OF 
LDAP FILTER; 

1) For complex ldap_filter with an AND operation: 
SET NEXTFILTER = LDAP-FILTERl 
LOOP UNTIL NEXTFILTER IS EMPTY 

SET NEXTFILTER - PROCESSFILTER 30 

(NEXTFILTER) 
CONCATENATE" INTERSECT" TO SQL- 
EXPRESSION. 
END OF LOOP 

2) For complex Idap-filter with an OR operation: 
SET NEXTFILTER - LDAP-FILTERl 
LOOP UNTIL NEXTFILTER IS EMPTY 

SET NEXTFILTER - PROCESSFILTER 

(NEXTFILTER) 
CONCATENATE" UNION" TO SQL-EXPRESSION. 
END OF LOOP 

3) For complex Idap-filter with a NOT operation: 
ADD " NOT " TO SQL-EXPRESSION. 

PROCESSFILTER (LDAP-FILTERl) 45 

4) For simple equality filter: 
CONCATENATE "SELECT ATTRIBUTE_TABLE_ 
NAME WHERE 

ATTRIBUTE_COLUMN„NAME = 'VALUE' " 
TO SQL-EXPRESSION. 

5) For simple substring filter: 
CONCATENATE "SELECT ATTRIBUTE_TABLE_ 
NAME WHERE 

ATTRIBUTE_COLUMN_NAME LIKE' SUB- 55 

STRING' " 
TO SQL-EXPRESSION. 

6) For simple greater or equal filter: 
CONCATENATE "SELECT ATTRIBUTE_TABLE_ 
NAME WHERE 

ArrRIBUTE_COLUMN_NAME>= 'VALUE' " 
TO SQL-EXPRESSION. 

7) For simple less or equal filter: 
CONCATENATE "SELECT ATTRIBUTE_TABLE_ 65 
NAME WHERE 

ArTRIBUTE_COLUMN_NAME <- 'VALUE' " 
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TO SQL-EXPRESSION. 

8) For simple attribute exists filter: 
CONCATENATE "SELECT ATTRIBUTE_TABLE_ 
NAME" 

5 TO SQL-EXPRESSION. 

9) For simple approximate filter: 
CONCATENATE "SELECT ATTRIBUTE_TABLE_ 
NAME WHERE 

SOUNDEX (ATrRIBUTE_COLUMN_NAME) = 

SOUNDEX ('VALUE')" 
TO SQL-EXPRESSION. 
Concatenate ")" to SQL- expression. 
Return next - Idap-filter. 
15 End processFilter algorithm. 

EXAMPLES 

With the basic translation rules and the EID sets approach 
described in the flowcharts of FIG. 5 and FIGS. 6A-6D, the 
20 following are the SQL queries that the invention generates 
for a representative LDAP filter query of the following form 
(|(fl='vr) (Q='v2')): 
Alternative 1 

SELECT entry.EntryData 
FROM LDAP_ENTRY as entry 
WHERE entry.EID in 
( 

SELECT distinct LDAP_ENTRY.E1D 
FROM ldap_entry, ldap_desc, fl 
WHERE 

(ldap_entry.EID=Idap_desc.DEID AND 
ldap_desc_AID-<id>) AND 
ldap„entry.eid=fl.eid AND 
fl='vl') 
UNION 

SELECT distinct ldap_entry.EID 
FROM ldap_entry, ldap_desc, £2 
WHERE (ldap_entry.EID=ldap+desc.DEID ADND 
ldap_„desc.AElD=<id>) 
AND Idap_entry.EID=f2.eid 
AND f2='v2*)) 
Alternative 2 

SELECT entry.EntryData 

FROM LDAP_ENTRY as entry WHERE entry.EID in 
( SELECT distinct LDAP„ENTRYEID FROM 
LDAP_ENTRY,ldap_desc 
WHERE 

(LDAP_ENTRY.EID=ldap_desc.DEID AND ldap_ 

desc.AEID=<id>) 
AND LDAP_ENTRY.EID 

IN ((SELECr EID FROM fl WHERE n = ' vl') 
UNION (SELECT EID FROM SN WHERE SN =' v2' ))) 
Both SQL statements illustrated above generate the cor- 
rect search results, and both techniques are within the scope 
of the present invention. As can be seen, the first query 
performs the JOIN operation with the Idap descendant table 
within each subquery. The second query performs the JOIN 
operation with the Idap descendant table outside the sub- 
query. Although either alternative may be used to implement 
the present invention. Alternative 2 may provide better 
performance results. In addition to correct results, the OR 
operation (illustrated above) performs well with both alter- 
native techniques because irrelevant entries are filtered out 
in the subquery and target entries are reported back to the 
main query. 
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As noted above, with the set-based approach of the 
present invention, the LDAP NOT operation preferably is 
performed by excluding entries through negating the IN 
operation before the subquery. The following example illus- 
trates the operation: 5 

Filter String: 

(!(fl='vr)) 

SQL Statement: 

SELECT entry.EntryData, ^° 
FROM LDAP_ENTRY as entry 
WHERE entry.EID in 

( SELECT distinct LDAP_ENTRY.EID FROM 

LDAP_ENTRY.ldap_desc 15 
WHERE (LDAP_ENTRY.EID-ldap_desc.DEID AND 

ldap_desc.AEID=<id>) 
AND LDAP_ENTRY.EID NOT IN ((SELECT EID 

FROM fl 20 

where fl=' vl'))) 
The following is another example of a SQL statement 
generated for complex query with AND, OR and NOT 
operator. 

Complex Query with AND, OR and NOT Operator ^5 
Filter String: 

(&(| (objcctclass-PERSON) (objectclass=GROUP)) (sn= 
SMITH) (!(member=*))) 

30 

SQL Statement: 

SELECT entry.EntryData, 

FROM LDAP_ENTRY as entry WHERE entry.ElD in 
( SELECT distinct LDAP_ENTRY.EID FROM 

LDAP_ENTRY,ldap_dcsc 
WHERE (LDAP..„ENTRYEID=ldap_desc.DEID AND 

ldap_desc.AElb=?) AND 
LDAP_ENTRY.EID 40 
IN (((SELECT EID FROM OBJECTCLASS WHERE 

OBJECTCLASS = 
PERSON) 

UNION (SELECT EID FROM OBJECTCLASS 

WHERE OBJECTCLASS = 
GROUP)) 

INTERSECT (SELECT EID FROM SN WHERE SN = 
SMITH ) 

INTERSECT 50 
(SELECT EID FROM LDAP_ENTRY WHERE EID 
NOT IN 

(SELECT EID FROM MEMBER)))) 

It should be noted that the subtree search level is illus- 
trated in the examples herein, but only for purposes of 55 
illustration. This should not be taken by way of limitation. 

As noted above, the invention may be implemented in any 
hierarchical directory service in which a relational database 
management system (RDBMS) is used to provide a backing 
store function. Thus, for example, the principles of the 60 
invention may be carried out in an X.500 directory service 
or hereinafter-developed LDAP implementations. The SQL 
query generated according to the present invention is used to 
access the relational database, and results are then returned 
in response to this query. 'Vb& invention may also be imple- 65 
mented within a relational database management system 
being used as an add-on to a directory service. One of 
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ordinary skill will appreciate that the invention can be 
applied to any relational database management system 
(RDBMS) and not simply DB/2, the implementation 
described above. Thus, for example, the relational database 
may be Oracle, Sybase or any other third party supplied 
backing store. In addition, the EID sets approach can also be 
applied to b-tree based LDAP server implementation. 

Moreover, although the preferred embodiment has been 
described in the context of generating a Structured Query 
Language (SQL) query, the inventive technique should be 
broadly construed to extend to any relational database query 
language. 

One of the preferred embodiments of the routines of this 
invention is as a set of instructions (computer program code) 
in a code module resident in or downloadable to the random 
access memory of a computer. 

Having thus described our invention, what we claim as 
new and desire to secure by Letters Patent is set forth in the 
following claims. 

What is claimed is: 

1. A method for searching a relational database using 
hierarchical, filter-based queries, comprising the steps of: 

parsing a filter-based query for elements and logical 
operators of the filter query; 

for each filter element, generating an SQL subquery 
according to a set of translation rules; 

for each SQL subquery, generating a set of entry ID's for 
the filter query; and 

combining the SQL subqueries into a single SQL query 
according to a set of combination rules chosen corre- 
sponding to the logical operators of the filter query. 

2. The method as described in claim 1 wherein the 
filter-based query is a Lightweight Directory Access Proto- 
col (LDAP) directory service query. 

3. The method as described in claim 2 wherein the logical 
operators of the LDAP filter-based query include AND, OR 
and NOT. 

4. The method as described in claim 3 wherein the 
combination rules map the OR logical operator to an SQL 
UNION operator. 

5. The method as described in claim 3 wherein the 
combination rules map the AND logical operator to an SQL 
INTERCEPT operator. 

6. The method as described in claim 3 wherein the 
combination rules map the NOT logical operator to an vSQI. 
NOT IN operator. 

7. The method as described in claim 2 wherein the 
relational database is DB/2. 

8. The method as described in claim 1 further including 
the step of accessing the relational database using the single 
SQL query, 

9. The method as described in claim 8 further including 
the step of returning a response to the single SQL query. 

10. The method as described in claim 1 wherein the step 
of combining the SQL subqueries is carried out recursively 
until all filter elements of the filter query have been pro- 
cessed into the SQL query. 

11. A method for searching a relational database from a 
Lightweight Directory Access Protocol (LDAP) directory 
service generating filter-based queries, comprising the steps 
of: 

parsing an LDAP filter-based query for elements and 
logical operators of the LDAP filter query; 

for each LDAP filter element, generating an SQL sub- 
query according to a set of translation rules; 

for each SQL subquery, generating a set of entry lO's for 
the LDAP filter query; and 
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combining the SQL subqueries into a single SQL query 
according to a set of conabinalion rules chosen corre- 
sponding to the logical operators of the LDAP filter 
query. 

12. The method as described in claim 11 wherein the 5 
logical operators of the LDAP filter-based query include 
AND, OR and NOT. 

13. The method as described in claim 12 wherein the 
combination rules map the OR logical operator to an SQL 
UNION operator. lO 

14. The method as described in claim 12 wherein the 
combination rules map the AND logical operator to an SQL 
INTERCEPT operator. 

15. The method as described in claim 12 wherein the 
combination rules map the NOT logical operator to an SQL 15 
NOT IN operator. 

16. The method as described in claim 11 wherein the step 
of combining the SQL subqueries is carried out recursively 
until all filter elements of the filter query have been pro- 
cessed into the SQL query. 20 

17. A computer program product in computer-readable 
media for searching a relational database using hierarchical, 
filter-based queries, comprising: 

means for parsing a filter-based query for elements and 
logical operators of the filter query; 25 

means for generating an SQL subquery for each filter 
element according to a set of translation rules; 

means for generating a set of entry ID's for each SQL 
subquery; and 

means for combining the SQL subqueries into a single 
SQL query according to a set of combination rules 
chosen corresponding to the logical operators of the 
filter query. 

18. The computer program product as described in claim 35 

17 wherein the filter-based query is a Lightweight Directory 
Access Protocol (LDAP) directory service query. 

19. The computer program product as described in claim 

18 wherein the logical operators of the LDAP filter-based 
query include AND, OR and NOT. 40 

20. The computer program product as described in claim 

19 wherein the combination rules map the OR logical 
operator to an SQL UNION operator. 

21. The computer program product as described in claim 
19 wherein the combination rules map the AND logical 45 
operator to an SQL INTERCEPT operator. 

22. The computer program product as described in claim 
19 wherein the combination rules map the NOT logical 
operator to an SQL NOT IN operator. 

23. The computer program product as described in claim 
18 wherein the relational database is DB/2. 
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24. A directory service, comprising: 

a directory organized as a naming hierarchy having a 
plurality of entries each represented by a unique iden- 
tifier; 

a relational database management system having a back- 
ing store for storing directory data; 
means for searching the directory, comprising: 

means for parsing an hierarchical, filter-based query for 

elements and logical operators of the filter query; 
means for generating a relational database subquery for 
each filter element according to a set of translation 
rules; 

means for generating a set of unique identifiers for each 

relational database subquery; and 
means for combining the relational database subqueries 

into a single relational database query according to a 

set of combination rules chosen corresponding to the 

logical operators of the filter query. 

25. The directory service as described in claim 24 wherein 
the directory is compliant with the Lightweight Directory 
Access Protocol (LDAP). 

26. The directory service as described in claim 25 wherein 
the relational database management system is DB/2. 

27. In a directory service having a directory organized as 
a naming hierarchy, the hierarchy including a plurality of 
entries each represented by a unique identifier, the improve- 
ment comprising: 

a relational database management system having a back- 
ing store for storing directory data; 
means for searching the directory, comprising; 

means for parsing an hierarchical, filter-based query for 

elements and logical operators of the filter query; 
means for generating a relational database subquery for 
each filter element according to a set of translation 
rules; 

means for generating a set of unique identifiers for each 

relational database subquery; and 
means for combining the relational database subqueries 

into a single relational database query according to a 

set of combination rules chosen corresponding to the 

logical operators of the filter query. 

28. In the directory service as described in claim 27 
wherein the directory is compliant with the Lightweight 
Directory Access Protocol (LDAP). 

29. In the directory service as described in claim 28 
wherein the relational database management system is 
DB/2. 

))( ^ i{c 
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ABSTRACT 



The semantics of the outer join operator are extended to 
permit the application of different predicates to the join 
tuples and the anti-join tuples. For un-nesting of nested 
query blocks, the anti-join tuples, for example, are asso- 
ciated with a count value of zero instead of a count 
value of null. An inner query block is un-nested from an 
outer query block by converting the inner query to a 
first un-nested query generating a temporary relation 
and converting the outer query block to a second un- 
nested query receiving the precomputed temporary 
relation. When the nested inner query has an equi-join 
predicate joining a relation of the inner query to an 
outer query and a count aggregate, the query blocks are 
un-nested by removing the equi-join predicate from the 
inner query and placing a corresponding conjunctive 
(left) outerjoin predicate term in the predicate of the 
outer query, performing the count aggregate for each 
distinct value of the joining attribute of the relation of 
the inner query, and in the outer query applying differ- 
ent predicates to the joining and anti-joining tuples such 
that the predicate of the anti-joining tuples is evaluated 
assuming a count value of zero. 

20 Claims, 18 Drawing Sheets 
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EXTENDING THE SEMANTICS OF THE OUTER 
JOIN OPERATOR FOR UN-NESTING QUERIES 
TO A DATA BASE 

5 

BACKGROUND OF THE INVENTION 

1. Technical Field 

The present invention relates generally to database 
management systems, and more particularly to query 
optimization in a database system. The present inven- 
tion specifically relates to un-nesting nested database 
queries by distinguishing between joining and anti-join- 
ing tuples d;iring an outer join operation. 

2. Background of the Invention 

A database management system is a computer system 
that provides for the storage and retrieval of informa- 
tion about a subject domain. Typical examples are air- 
line reservation systems, payroll systems, and inventory 
systems. The database management system includes an 
organized collection of data about the subject domain, 
and a data-manipulation language for querying and 
altering the data. Typically, the data are organized as 
"tuples" of respective values for predefined attributes. 

In a so-called "relational" database management sys- 
tem, the tuples of data are stored in a plurality of tables, 25 
each of which corresponds to a set of tuples for com- 
mon predefined attributes. Each table corresponds to a 
single "relation". Each attribute of the relation corre- 
sponds to a column of the table, and each tuple of values 
corresponds to a particular row of the table. Each value 30 
in the table corresponds to a particular row or tuple and 
a particular column or attribute. A collection of related 
tables or relations in the database is known as a schema. 

The data manipulation language for a relational data- 
base management system typically specifies operations 35 
upon one or more relations to create a new relation. A 
"restriction" operation forms a subset of the rows in a 
table by applying a condition or "predicate" to the 
values in each row. A "projection" operation removes 
columns from the rows by forming a stream of rows 40 
with only specified colunms. A "join" operation com- 
bines data from a first table with data from a second 
table, typically by applying a predicate comparing the 
values in a column of the first table to the values in a 
related colunm of the second table. Usually such a join 45 
predicate is an "equi-join" predicate in which a first 
function of the attribute corresponding to the column of 
the first table must equal a second function of the attri- 
bute corresponding to the column of the second table. 

Yet another operation upon a relation is known as an 50 
aggregation, in which a new column of values is gener- 
ated by combining or aggregating values from all of the 
rows, or specified groups of rows. Aggregate functions 
include, for example, a count of the rows in each group, 
or the sum of the values of a specified attribute for all 55 
rows in each group. Typically, the rows are grouped for 
aggregation such that the rows in each group have 
equal values for a specified column or attribute. There- 
fore, a new aggregate value is generated corresponding 
to each distinct value of the specified attribute. 60 

The query language for a relational database system 
typically defines a syntax for specifying a "query block" 
including a list of relations to be accessed, a predicate to 
govern restriction or join operations, a list of attributes 
to specify a projection operation, and a list of aggregate 65 
functions. If at least one attribute is specified and at least 
one aggregate function is also specified, then a meaning- 
ful result usually would require the aggregate to be 
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grouped for each distinct value of the specified attri- 
bute. Although such a grouping could be presumed, the 
query language may permit or require grouping in this 
situation to be specified by a list of grouping attributes. 
In this case, the rows are grouped prior to aggregation 
such that each group corresponds to each distinct com- 
bination of values for all of the specified grouping attri- 
butes. 

Some query langtxages permit query blocks to be 
nested such that the predicate of an "outer" query block 
includes reference to the result of an "inner" query 
block. In this case, the query language specifies a result 
that would be obtained by re-evaluating the inner query 
block each time the outer query block evaluates the 
predicate for a different row or combination of rows 
when performing its specified restriction or join opera- 
tions. Evaluating the result in such a fashion, however, 
usually is very inefficient. 

To select the most efficient of alternative ways of 
evaluating a query, the database management system 
typically includes a query optimizer. In general, a query 
can be evaluated a number of ways, because, in many 
cases, the query operations obey certain commutative, 
associative, or distributive laws. By applying these laws, 
the query optimizer may formulate alternative orders of 
performing the query operations, compute a cost of 
performing each such "query plan", and select the least 
costly query plan for execution. 

Unfortunately, if a query is specified by nested query 
blocks, the nesting itself specifies an iterative order of 
performing query operations. Since the query opera- 
tions of an inner block are specified to occur during the 
restriction or join operation of the outer query block, 
the query optimizer cannot apply the laws permitting 
alternative execution orders unless the nested query 
blocks are first "un-nested" into equivalent "pipelined" 
query blocks. 

' When query blocks are pipelined, the result of a first 
query block is specified as input to the predicate of a 
second query block, but that result is presumed to be 
evaluated only once before evaluation of the second 
query block. Therefore, there is a distinct order of per- 
formance specified between the first query block and 
the second query block. In many cases the un-nesting 
process generates un-nested query blocks in which this 
specified order is more efficient that the iterative 
method of executing the original nested query blocks. 
In other cases, the optimizer can find an even better 
order of execution for evaluating the un-nested query 
blocks. 

The problem of un-nesting and optimizing database 
queries has been studied extensively. A general solution 
was proposed in Won Earn, "On Optimizing an SQL- 
like Nested Query", ACM Transactions on Database 
Systems, Vol. 9, No. 3, American Association for Com- 
puting Machinery, United States (1982), incorporated 
herein by reference. Later, it was discovered that the 
un-nesting technique of Kim does not always yield the 
correct results for nested queries that have non equi-join 
correlation predicates or that have a "COUNT" aggre- 
gate between the nested blocks. Un-nesting solutions for 
these anomalous cases were described in Richard A. 
Ganski and Harry K. T. Wong, "Optimization of 
Nested SQL Queries Revisited", Proceedings of the 
Sigmod Conference, American Association for Com- 
puting Machinery, United States (1987), pp. 23-33, 
incorporated herein by reference. Ganski's method of 



un-nesting was extended to multiple nested blocks as 
disclosed in Umeshwar Dayal, "Of Nests and Trees: A 
Unified Approach to Processing Queries That Contain 
Nested Subqueries, Aggregates and Quantifiers**, Pro- 
ceedings of the 13 VLDB Conference, Brighton, 1987, 
pp. 197-208, incorporated herein by reference. Methods 
of un-nesting multiple blocks within the same block are 
disclosed in M. Muralikrishna, "Optimization and Data- 
flow Algorithms for Nested Tree Queries**, Proc. 
VLDB Conf. (August 1989), pp. 77-85, incorporated 
herein by reference. The commuting of joins and outer 
joins imder specific conditions is disclosed in Amon 
Rosenthal and Cesar Galindo-Legaria, "Query Graphs, 
Implementing Trees, and Fieely-Reorderable Outer- 
joms'*, Proc. SIGMOD Conf (May 1990), pp. 291-299, 
incorporated herein by reference. 

SUMMARY OF THE INVENTION 

Briefly, in accordance with the present invention, 
there is provided an alternative method of un-nesting 
queries with equi-join predicates and a "COUNT** ag- 
gregate between the nested blocks. This alternative is 
used where it is more efficient, than the method of Gan- 
ski, supra. The method of the present invention extends 
the semantics of the outer join operator to permit the 
application of different predicates to the join tuples and 
the anti-join tuples. The anti-join tuples, for example, 
are associated with a count value of zero. 

A further aspect of the present invention involves the 
use of the novel un-nesting method to queries with 
multiple blocks. In this case, the alternative un-nesting 
method of the present invention is used when the corre- 
lation predicates in the Count block are "neighbor pred- 
icate*' referencing the relation in their own block and 
the relation from the immediately enclosing block. Oth- 
erwise, the Ganski-Dayal un-nesting method is used. 

StiU another aspect of the present invention is an 
integrated procedure for evaluating joins and outer- 
joins in a top-down order. This integrated procedure 
enables the alternative un-nesting method of the present 
invention to be applied to more blocks. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Other objects and advantages of the invention will 
become apparent upon reading the following detailed 
description and upon reference to the drawings in 
which: 

FIG. 1 is a block diagram of a database management 
system incorporating the present invention; 

FIG. 2 is a block diagram of a relational database 
used in the database management system of FIG. 1; 

FIG. 3 is a flowchart of the basic steps followed by 
the database management system of FIG. 1 for process- 
ing a query; 

FIG. 4 is a block diagram of a query node generated 
during the parsing of a query block by the database 
management system; 

FIG. 5 is a flowchart of a procedure followed by the 
database management system of FIG. 1 when parsing a 
query block; 

FIG. 6 is a flowchart of a procedure used by the 
database management system when parsing a predicate 
in a query block; 

FIG. 7 is a flowchart of a procedure for executmg a 
query block; 

FIG. 8 is a detailed flowchart of a specific implemen- 
tation of the flowchart of FIG. 7 in accordance with 
tuple iteration semantics; 
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FIG. 9 is a data flow graph representation of two 
nested query blocks; 

FIG. 10 is a data flow representation of two pipelined 
query blocks obtained by applying Kim's un-nesting 
5 method to the nested query blocks of FIG. 9; 

FIG. 11 is a data flow representation of two pipelined 
data blocks obtained for a count aggregate when the 
nested query blocks of FIG. 9 are un-nested by Ganski*s 
method; 

10 FIG. 12 Is a data flow representation of two pipelined 
query blocks obtained for a connt aggregate when the 
nested query blocks of FIG. 9 are un-nested by the 
method of the present invention; 
FIG. 13 is a flowchart of a specific procedure for 

15 performing a left outer join operation upon a count 
aggregate and applying difierent predicates to the join- 
ing and anti-joining tuples; 

FIG. 14 is a join graph used by the query optimizer of 
the present invention; 

20 FIGS. 15 and 16 together comprise a flowchart of an 
integrated un-nesting procedure in accordance with the 
present invention; 

FIGS. 17 to 22 are six alternative query graphs ob- 
tained by un-nesting the query graph shown in FIG. 14 

25 in accordance with the procedure of FIGS. 15 and 16. 
While the invention is susceptible to various modifi- 
cations and alternative forms, a specific embodiment 
thereof has been shown by way of example in the draw- 
ings and will herein be described in detail. It should be 

30 understood, however, that it is not intended to limit the 
invention to the particular form disclosed, but, on the 
contrary, the intention is to cover all modifications, 
equivalents, and alternatives falling within the spirit and 
scope of the invention as defined by the appended 

35 claims. 

DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENT 

Turning now to FIG. 1 of the drawings, there is 

40 shown a database management system generally desig- 
nated 30. The database management system 30 is made 
by programming a conventional digital computer. The 
database management system 30 includes a memory 31 
storing a relational data-base 32, as will be further de- 

45 scribed below with reference to FIG. 2. For processing 
data in the relational data-base 32, the database manage- 
ment system 30 includes a data processor 33. For com- 
municating with a user 34, the database management 
system includes an input device 35, such as a keyboard, 

50 and an output device 36, such as a video display. 

The present invention more particularly concerns the 
processing of a query from the user 34 instructing the 
database management system 30- to display to the user 
specified data from the relational database 32, The 

55 query is received by the mput device 35 and processed 
by a query processing system 37 in the data processor 
33. In particular, the query processing system is made 
by loading a computer program into the data processor 
33 and executing that computer program. The query 

60 processing system 37 includes a parser 38 which con- 
verts the query from the user 34 into an internal repre- 
sentation that identifies and locates specific components 
of the query. The internal representation, for example, 
is a hierarchical graph constituting one of the query 

65 graphs 39 stored m the memory 31. 

The query graph of the parsed query specifies a par- 
ticular order of execution, but the specified order of 
execution is not necessarily the optimum way of execut- 
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ing the query. Instead, there may be any number of or more columns of the table as including keys to the 

better alternative ways of executing the query to obtain individual rows. Some operating systems, for example, 

identical results. To possibly obtain a more optimum have optimized procedures for storing tables in files 

execution plan, the query processing system 37 mcludes according to a primary key and for retrievmg a row or 

a query optimizer 40 that operates upon the query graph 5 record specified by the primary key. In such systems, if 

of the parsed query to generate a number of alternative the user does not specify a column as including the 

query plans, each specified by a respective one of the primary key, the system may nevertheless assign a pri- 

query graphs 39. The query optimizer 40 computes a mary key to each row. In such a case, the primary key 

cost associated with each of the alternative query plans, is known as a "surrogate column" which, if not accessi- 

and executes the query plan having the least cost. Dur- 10 ble to the user, will be accessible to the computer pro- 

ing execution of this query plan, a retrieval system 41 grammer. 

accessing data from the relation database 32, and per- Turning now to FIG. 3, there is shown a flowchart of 

forms upon the data a number of relational operations. the basic procedure followed by the query processing 

These relational operations result in an output relation system (37 in FIG. 1) when processing a query. In a first 

which is transmitted to the output device 36 and dis- 15 step 61, a root node is cleared for a query graph to 

played to the tiser 34. receive a query from the user 34. Next, in step 62, a 

Turning now to FIG. 2, there is shown a detailed current node pointer is set to point to the root node, 

representation of the relational database 32. The rela- Then, in step 63, a query is received from the user 34, 

tional database 32 includes data stored in a number of The query is a string of alphanumeric characters 

tables, such as the tables 51 and 52. Each table corre- 20 which conform to a query language that identifies data 

spends to a particular relation in the relational database. in the relational database and operations to perform 

The table 51, for example, corresponds to a relation "R" upon the data. In many conventional query languages, 

and the table 52 corresponds to a relation "S". Each the query is in the form of a query block, which may 

table is shown as a rectangular matrix of rows and col- refer to additional nested query blocks, as will be fur- 

imms. Internally, however, the data could be stored in 25 ther described below. 

any number of ways. Each row, for example, could be To process the query, the query is converted or 
stored as a record of a file. Alternatively, the data in parsed from the variable-length format of the query 
each of the tables could be freely dispersed in memory, language into a more fixed format for processing by the 
and the rows and columns could be defined by lists of query optimizer (40 in FIG, 1) and execution by the 
pointers to the data. In any event, the data structures 30 retrieval system (41 in FIG. 1). The query block is 
storing the data in the tables 51 and 52 permit all of the parsed in step 64, for example, by calling a subroutine 
rows in a table to* be accessed sequentially and further further described below with reference to FIG. 5. Pars- 
permit a specified column position of an accessed row ing of the query block generates a node in the query 
to be indexed. graph. If the parsed query block includes nested nodes. 

Each coltimn in each of the tables is indexed by a key 35 the nested nodes are parsed recursively, creating addi- 

called an attribute. Most conventional database query tional nodes in the query graph that are linked to the 

languages assign an alphanumeric relation name to eacli root node. Once the query block is parsed, execution 

table and further assign an alphanumeric attribute name continues from step 64 to step 65. 

to each colunm of each table, but the attribute name If the query from the user fails to conform to the 
associated with one table need not be distinct from the , 40 syntax of the query language, the parsing in step 64 will 

attribute name of another table. A particular colunm of return with an error. If so, then further processing of 

data, however, can be uniquely specified by the combi- the query cannot continue, and execution therefore 

nation or concatenation of both a relation name and an returns, so that the user may submit a corrected query, 

attribute name. Otherwise, execution continues in step 66 by optimizing 

To index the columns of the tables, the relational 45 the query, 

database 32 further includes a number of indices, or Query optimization is further described below with 

definitions of keys, to the tables and columns. These reference to FIGS. 9 to 22. The query optimization 

indices are known as a schema 53 corresponding to a results in a number of alternative query plans repre- 

particular subject domain. The schema 53 is further sented by respective query graphs (39 in FIG, 1). The 

subdivided into an mdex for each relation, such as an 50 optimizer (40 m FIG. 1) selects the query plan having 

index 54 to the relation "R" of the first table 51, and an the least computational cost, and, in step 67, that query 

index 55 to the relation "S" for the second table 52. The plan is executed. The query plans includes nodes, each 

index 54 to the relation "R" includes a name ("R") for of which corresponds to the operations of a single query 

the relation and a name ("a", "b**, "c", "d", "e", "f block. The execution of a single query block is further 

"g") for each attribute corresponding to each colunm of 55 described below v^th reference to FIGS, 7, 8 and 13, 

the table R, and the index 55 to the relation "S" includes For the sake of illustration, the syntax for a specific 

a name C'S") for the relation and a name ("h", "i", "j", query language will now be described. This syntax will 

"k", "1") for each attribute corresponding to each col- roughly correspond to "Structared Query Language" 

umn of the table 52. The indices 54 and 55, for example, (SQL) as adopted by the American National Standards 

permit a numerical pointer or address to be obtained to 60 Institute in 1986. The use of such a standard query lan- 

a table and column specified by a relation name and an guage in the context of a relational database system is 

attribute name. described in Hobbs and England, Rdb/VMS—A Com- 

The indices 54 and 55 may further include respective prehensive Guide, Digital Equipment Corporation, May- 
indices 56 and 57 to the individual rows of the tables. nard, Mass. (1991), incorporated by reference, on pages 
For processing most relational operations, it is only 65 36-64. 

necessary to scan sequentially through all of the rows of It is possible to represent virtually any query by a 

a table. To facilitate the selection and retrieval of a pipelined series of query blocks, each of which specifies 

particular row, however, it is desirable to identify one a list of relations, a predicate for restricting or joining 
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rows of the relations, a list of columns to be selected as -continued 
output, and a list of aggregate functions. If at least one 



column and at least one aggregate function is specified, °P""|"| [Relational. operator] [tenn] 

. , , . , t 1 . 1 [Logical operator] [term] [Relational operator] [term] 

it IS also desirable to specify groups of rows over which 
the aggregate functions are to be performed. Typically, 5 

it is desurable to group rows having identical values in ^. v^v^^,., 

specified columns. Therefore, regardless of the particu- Tunung now to FIG. 5, there is shown a flowchart of 

lar syntax of the query language, virtually any query to f procedure for parsmg a query block havmg the above 

a relational database can be represented as a graph of f^™^*. In a f^st step 91, a first wordm the query is 

nodes having a format similar to the format shown in 10 P^^ed If the first word is not SELECT , as tested m 

FIG 4 ^^^P ' ^ syntax error has occurred. The user is 

A t. • T-T/-^ A J on • I J notified of the error in step 93, and execution returns. 

As shown m FIG. 4, a query node 80 includes a "w""*^" ^hwa ovt.|^ vavv.ui.iv/a/ xv^v^uo. 

pointer 81 to its parent node in the query graph, an " ^L- Z"."^' "^TV^ 

attribute list 82 specifying the columns in S output Pf^^^^" ^^J^l'^ f "FROM", as t^ted m step 95, 
relation, a "GROUPBY" list 83 specifying the columns 15 word should be part of the attnbute hst or the 

for grouping rows having identical values in the col- ^Sgregate list The word is compared to reserved words 

umns, an aggregate list 84 of the aggregate functions to ^^ri^ ^*^^ f^T"^ ^ -'"f ^ 

perform upon the grouped rows, a relation list 85 sped- COUNT or SUM), and if Ae word is recogmzed as an 

fying the kiput tolhe query node, a predicate 86 to be aggregate, it is placed m the aggregate hst. Otherwise, 
tested against combinations of rows of the input rela- 20 the word is placed m the attnbute hst. If the word does 

r* *^^r. QT * ^^s*^^ +1,^ ™. uot correspond to an aiphanumenc name for an attn- 

tions, pointers 87 to nested children of the query node . , ^ ^ ^ ^ . 

on c« «;««^t;«^^ Dute, then an error has occurred and execution returns. 

80, and pomters 88 to pipelmed children oi the query ' . . , „ „ , , , ^ 

node 80 - ^ Otherwise, execution loops from step 96 back to step 94 

° The hierarchical" structure of a query graph including ^^.P^^^ the next word in the query. Eventually step 95 
the query node 80 specifies an order of execution among 25 will recogmze the word FROM" begmmng tiie rela- 

the query nodes. According to the specified order, the ^ f P °f f P^^^/^ '° 

, ^, u • c X ■ * J ^ J 4. 98, the word is compared to WHERE . If the word is 

root node of the query graph is first mspected to deter- * <TTT7TTTr,^Tf» u i^u i +■ -ru r 

1, *!, •+ vT +™ ; not WHERE", then it should be a relation. Therefore, 

mme whether it has any pomters to pipelmed children. • . \ ^ ^ t ^ rux ^ 

Tf *t. *i. • r A wAA^^ ^r.A^^ i^r> ^^+r.A execution branches to step 99, In step 99, execution 

If so, then the pipelmed children nodes are mspected m , , ^ ^ -.n^ -x-*!. ^/^t^V^tt^tdv" 

. r -L • J • f J t- • • 30 branches to step 103 if the word is GROUPBY . Oth- 

a recursive fashion until a node IS found having no pipe- . . . . ^nn t * -mn 

V J T-u- ^ 11 J «i^>.f» • erwise, execution contmues to step 100. In step 100, 

lined children. This so-called leaf node m the query * ^ . - ^ ^ Tr 

, . . , , - , „ -^^ ^ execution returns if the end of the query is reached. If 

graph IS executed, which may require the execution or ^ . . ^- ^ 7 ^ 

® f J ,,1 'f. A -u *\, • * +^ not, then execution continues to step 101 to add the 

nested query blocks specified by the pomters to the j ^ .i. i v * Ci i t*- 4.1, a 

* J t,-iJl r\ f , word to the relation list of the query block. If the word 

nested children. Once execution of a query node is , , ^ r j ir j i 

, ^ J ^ . V : J T 35 does not correspond to the name of a predefined rela- 

completed, execution returns to its parent node. In this . - ^ . ^ ^ ^ 

^lt- j j r ' r *u u tion, however, an error has occurred, and execution 

way, the specified order of execution IS from the bottom • i i_ i * * n-r 

of the que^ graph to the top of the query graph. Once ^^'""'f " ^'^^TT'.uTT'' P" li, 
the root query node is executed, the resulting relation is ^^^P ^nds that the word is WHERE , then 
transmitted as output to the user execution contmues to step 102 to parse the predicate of 
The execution order specified by the query graph 'T^^" '^^iT^^'^i' P^j^'^f 
might not be the best order of execution Therefore, ^^'^^^^ "'^'o^ « ^ ^^^^ Pf f V^^c^^. 
when a query block is parsed, the parser (38 in FIG. 1) f.^f^'J^l^i^n v1 ?? ^ -^i^^i," ^ 
may identify node-type information 89 for use by the "GROUPBY" statement foUows the "WHERE" state- 
query optin^er (40 in FIG. 1). The node-type informa- '^^''l f ""f' *e end of the query block has been 
tion 89, for example, specifies whether the node has an f execution returns^ Othei^e m step 103 
aggregate function, whether the aggregate function is a foUowmg words are parsed mto the GROUPBY 
count function, whether the predicate is an equi-join ' , ^^^^ • i j t_ 
predicate, whether the node is a nested node of a parent . procedure of FIG 5 ,s also used when parsmg an 

node, and whether a nested node references a relation in °f ^t-T"?! v,^f ''T'/ . 'a 

a parent or ancestor node in which it is nested. '° ^^«°"^ff ^^''K f°''^^^ ^ -^^ . 1°' f 

For describing the parser 38, it is necessary to assume ¥^ ^^^^'^ ""ff f P^^°^ 

a specific format for a query from the user. A query will >° W graph. Step 105 is called when the predi- 

include, for example, a "SELECT" statement, a cate of an outer query block is parsed and the w^^^ 

"FROM" statement, possibly a "WHERE" statement, ,^^^^9^ 1^ ^ *f ™ j?^'^'?^*^' ^ 

and possibly a "GROUPBY" statement, arranged as further descnbed below with reference to FIG. 6 After 
follows- ' execution contmues m step 94 to parse the next 

SELECT [Attribute list] [Aggregate list] word in the inner nested query block. 
FROM TRelation listl Tummg now to FIG. 6, there is shown a flowchart of 

rWHERE rPredicate 11 ^ procedure for parsing the predicate. In a first step 116, 
[GROUPBY FAttribute listll ^ ^ °^ predicate is parsed. The term should 

When there is a "WHERE" statement having a predi- ^^^"^ ^« a constant, a predefined attribute, the end of 
cate, the predicate should have two terms jomed by a ^"^^ ^}°''^' *f f a GROUPBY state- 

relational operator, and possibly one or more logical °' /ff^^ specifymg a nested 

operators joining similar combinations of tenns as fol- ^ 'J"^^ » end of the query block is reached, as 

lows* ^ 65 tested m step 117, execution returns. If the term is 

GROUPBY, as tested in step 118, execution also returns 

If the term is "SELECT" as tested in 119, then in step 

[term] [Relation Operator] [term] 120, the procedure of FIG. 5 is called recursively to 
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parse the nested query block. In any case, execution ways. A fairly straightforward implementation is 

continues in step 121 to parse a comparison operator. If known as "tuple iteration" because it executes the query 

a predefmed comparison operator is not found, then an block by sequentially indexing the rows of the relations 

error has occurred and execution returns. Otherwise, in in the relation list so that all combinations of rows are 

step 122, a next term is parsed. If the term is "SE- 5 indexed. 

LECT", as tested in step 123, then the term is a nested Turning now to FIG. 8, there is shown a flowchart of 

query block, and execution branches to step 124 to a procedure for evaluating a query block by tuple itera- 

recursively call the procedure of FIG. 5 to parse the tion. In a first step 151, a row pointer P/ is allocated to 

nested query block. In any case, execution continues in each relation R/in the relation list. Then, in step 152, all 

step 125 to compare the type of the two previous parsed 10 of the row pointers are set to zero, and a variable NRO, 

terms to the type required by the comparison operator which is used to indicate the number of output rows, is 

parsed in step 121. If the type of the terms do not agree also set to zero. At this point, iteration may begin. In 

with the comparison operator, then an error has oc- step 153, execution branches depending on whether the 

curred and execution returns. Otherwise, execution predicate is null. If the predicate is null, then execution 

continues to step 126. Step 126 attempts to parse a logi- 15 branches to step 154. In step 154, execution branches 

cal operator. The predicate, however, might not have a depending on whether there is an empty aggregate list, 

logical operator. Step 127 determines whether the end If the aggregate list is empty, then in step 155, a row of 

of the query block has been reached. If so, execution the output relation is formed with the columns specified 

returns. Otherwise, step 128 determines whether the by the attribute names in the attribute list. This is the 

beginning of a "GROUPBY" statement has been 20 end of the operations for a single iteration or combina- 

reached. If so, execution returns. Otherwise, assunung tion of rows from the relations in the relation list. Since 

that a valid logical operator has been parsed, execution this iteration generated a new output row, the variable 

continues to step 129, where a next term is parsed. Exe- NRO is incremented by one in step 155. 

cution then loops back to step 119, Therefore, addi- Execution continues in step 157 to begin a series of 

tional terms and comparison operators are parsed unto 25 tests which determine the next combination of row 

the end of the predicate is reached in step 127 or 128. pointer values to index the next combination of input 

Specific programming for parsing expressions and rows. In step 157, an index I is set to zero. Step 158 then 

evaluating expressions by symbolic execution is found checks whether the Ith row pointer has reached its 

in Hardy et al., U.S. Pat. No. 4,648,044, issued Mar. 3, maximum value of PMX(I). If so, then in step 159, this 

1987, incorporated herein by reference. 30 pointer is set to zero. Then, in step 160, the index I is 

Turning now to FIG. 7, there is shown a flowchart of compared to NR, which is one less than the number of 

a procedure for executing a query block. This proce- relations in the relation hst Step 160 checks whether 

dure follows the specified way of executing a SQL the index I has reached its maximxm:i value. If so, then 

query block having a "SELECT", "FROM", no further iteration is necessary. In step 161, the col- 

"WHERE" and possibly a "GROUPBY" statement. 35 umns of output relation which are specified by the attri- 

This particular procedure should be modified, for exam- bute list are selected for output together with any ag- 

ple as shown in FIG. 13, to perform the method of the gregates, and execution returns. Otherwise, in step 162, 

present invention, the index I is incremented, and execution loops back to 

In a first step 131 of FIG. 7, a new relation is formed step 158 to check the next row pointer. Eventually, a 

having rows which are all combinations of aU rows of 40 row pointer will be found in step 158 to be less than its 

all relations in the relations list. Then, m step 132, the maximum, or else all pointers will be at their maximum 

predicate is tested for each row of the new relation, and values and execution will return from step 160. If the Ith 

rows are discarded for which the predicate is false. In row pointer is found in step 158 to be less than its maxi- 

other words, step 132 performs a restriction operation mum value, then in step 163, that row pointer is incre- 

upon the new relation formed in step 131, In step 133, 45 mented by one and execution loops back to step 153 to 

execution branches depending upon whether the aggre- begin another iteration. 

gate list is empty. If there are not any aggregates speci- When step 153 determines that the predicate is not 

fied by the aggregate list, then in step 134, a projection null, then in step 171, the predicate is tested for values 

operation is performed upon the remaining rows of new from the rows Ri(P/). In the next step 172, execution 

relation by selecting the columns specified by the attri- 50 branches depending upon whether the predicate is true, 

bute list, and execution returns. If so, execution branches to step 154 to further process 

If step 133 determines that aggregates are specified, that row. Otherwise, execution continues in step 157. 
then execution branches to step 135. In step 135, execu- When step 154 determines that the aggregate list is 
tion branches depending on whether the "GROUPBY" not empty, then execution branches to step 173. In step 
list is empty. If the "GROUPBY" list is empty, then in 55 173, execution branches depending on whether the 
step 136, an aggregation is performed over all remaining "GROUPBY" list is empty. If the "GROUPBY" list is 
rows of the new relation. Then in step 137, the aggre- empty, then the input row (P/) is aggregated. The col- 
gates are selected for output, and execution returns. umns over which the input row is aggregated are speci- 

If step 135 finds that the GROUPBY hst is not empty, fied by attribute arguments of the aggregate functions, 

then in step 138, an aggregate is performed over groups 60 The result is a single aggregate row, and therefore in 

of rows having the same values in all of the columns step 175, the variable NRO is set to one. Execution then 

specified by the attribute names in the "GROUPBY" continues in step 157. 

list. Then, in step 139, the columns specified by the If step 173 determines that the "GROUPBY" list is 

attribute list are selected for output, together with the not empty, then execution branches to step 176 to deter- 

aggregates, and execution returns. 65 mine whether the input row (P/) is part of a new group, 

FIG. 7 defines the conceptual operations required for or whether it is part of an already existing output group, 

executing the query block. In practice, however, these This determination is made by scanning all of the exist- 

conceptual operations can be performed in various ing output rows. In step 176, an output row pointer RO 
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is set to zero. Then, in step 177, the output row pointer 
RO is compared to the number of output rows NRO. If 
the row pointer RO is equal to the number of output 
rows, then execution branches to step 178. In step 178, 
the input row (Pf) is aggregated with a new output row 5 
(RO). Execution then continues in step 155 to increase 
the number of output rows. 

If, in step 177, it was found that the row pointer RO 
has not yet reached the current number of output rows 
NRO, then in step 179, the rows (P^) of the input rela- ^0 
tion are compared to the rows (RO) of the output rela- 
tion in all of the columns specified by the attribute list. 
In this regard, the aggregation process forms a new 
relation that includes the columns specified by the attri- 
bute list, any columns specified by the "GROUPBY" 
list not included in the attribute list, and a column for 
each aggregate function. Any column specified by the 
"GROUPBY" list that is not included in the aggregate 
list is a temporary column used by the scanning proce- 
dure, but which is not selected for output in step 161 at 
the end of evaluation. In the next step 180, execution 
branches to step 181 if there is not a match. In step 181, 
the scan index RO is incremented and execution loops 
back through steps 177, 179, and 180 until either a 
match is found in step 180 or all of the rows are scanned, 
as tested in step 177. 

If step 180 finds a match, then in step 182, the input 
rows (Pi) are aggregated with the output row (RO) and 
execution then continues in step 157. 

The present invention more particularly concerns 
un-nesting queries.- Consider, for example, the following 
two-block query: 



EXAMPLE 1 



SELECT R,a 
FROMR 

WHERE R.b OPi (SELECT COXJNT (*) 
FROMS 

WHERE R.c = S.c) 



20 



35 



40 



In this and the following examples, relations are named 
by capital letters, and attributes are named by small 
letters. Attribute names will be fully qualified by their 
relation names. Therefore, **R.c" represents the attri- 
bute c of the relation R. 

A query graph of this query is shown in FIG. 9, A 
node 191 represents the operations of the inner query, 
and a node 192 represents the operations of the outer 5q 
query. In Example 1 above, the predicate (R.c = S.c) of 
the inner query references the relation R in the outer 
query. Therefore, the inner node cannot be precom- 
puted. 

The un-nesting procedure of Kim, supra,, transforms 55 
the above query into the following two un-nested 
queries: 
Query 1: 

TEMP (c, count) = 

SELECT S.c, COUNT (*) 60 
FROMS 
GROUPBY S.c 
Query 2: 
SELECT R.a 

FROM R, TEMP 65 
WHERE R.c=TEMP.c and R.b OPi TEMP.count 
The result of the first query is a temporary relation 
TEMP that is pipelined into the second query. The first 



query computes a count value associated with every 
distinct value in the c attribute of S. 

A graph of the un-nested queries is shown in FIG. 10. 
The first query is represented by a node 197, and the 
second query is represented by a node 198. The first 
query computes a count for each distinct value of S.c. 
Kim concluded that the second query of FIG. 10 would 
give the same result as the outer query of FIG, 9 be- 
cause the join predicate **R,c=TEMP.c" of node 194 
would select the same aggregate for each value of R.c 
as would be computed by the inner node 191 of FIG. 9. 
Unfortxmately, this conclusion incorrectly presumes 
that there is some value of S.c associated with each 
count computed by the inner query of FIG. 9 for each 
row of R. Therefore, Kim's method of FIG. 10 will not 
work correctly for a count aggregate as shown. Kim's 
method, however, will work correctly for other aggre- 
gates such as MIN, MAX, SUM and AVERAGE. 

Consider now the following specific example where 
the aggregate is the count function: 

EXAMPLE 2 



SELECT DEPT.id 
FROM DEPT 

WHERE DEPT.pcs > (SELECT COUNT (♦) 
FROM EMP 

WHERE EMP.dept . name = DEPT.name) 



This nested query would find all departments that have 
more work stations than employees. The in the 
argument list of the COUNT function denotes a count 
of all combinations of tuples that satisfy the predicate 
"EMP.dept__name= DEPT.name". If an attribute oc- 
curs as an argument of the COUNT function, then a 
count is made of all such tuples that have a non-null 
value for the specified attribute. 

Applying Khn's un-nesting method to the nested 
query of Example 2 produces the two following 
queries: 

TEMP (dept-name, count) = 

SELECT EMP.dept-name, COUNT (*) 

FROM EMP 

GROUPBY EMP.dept_name 

SELECT DEPT.id 

FROM DEPT, TEMP 

WHERE DEPT,name=TEMP.dept„name 

AND DEPT,pcs>TEMP.count 
If there is a new department with no employees, but at 
least one work station, then Kim*s method gives an 
incorrect result. In this case, the relation DEPT will 
include a tuple or row with DEPT.pcs>0, but the rela- 
tion EMP will not include any tuple or row "associ- 
ated" with that tuple or row. In more precise terms, the 
tuple with DEPT.pc > 0 of DEPT.pcs will not have any 
tuples in EMP that will be joined with it by the join 
correlation predicate DEPT.name = EMP.dept-name. 
Therefore, the WHERE statement of the original query 
will have its predicate satisfied by that tuple because 
DEPT.pcs will have a value >0 and COUNT (*) will 
have a value of 0. But the un-nested query will not have 
its predicate satisfied by that tuple because the join 
correlation predicate DEPT.name=TEMP.dep- 
t— name will not be true for that particular department. 

A solution to the so-called "count bug" was de- 
scribed in Ganski, supra. With respect to Example 1 
above, for the case of AGG being the COUNT func- 
tion, Ganski reasoned that a tuple r of R would be lost 
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after the join in ICim's second query if it does not join 
with any tuples of S. However, the COUNT associated 
with r is 0 and if R.b OPj 0 is true, tuple r should appear 
in the result. In order to preserve tuples in R that have 
no joining tuples in S, Ganski proposed that a (left) 
outer join (OJ) should be performed when the COUNT 
aggregate is present between two blocks. In this case, 
the un-nested query becomes: 
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Query 1: 

TEMP (c, count) = 
SELECT R.c, COUNT (S.c) 
FROM R.S 
WHERE R.C OJ S.C 
GROUPBY Rx 
Query 2: 
SELECT R-a 
FROM R, TEMP 
WHERE R-c = TEMP.C 

AND R.b OPi TEMP.count 

A corresponding query graph is shown in FIG. 11. 
The first query is represented by a node 195, and the 
second query is represented by a note 196. The (left) 
outer join operation (OJ) preserves every tuple of R, 
and consequently a count is computed for every distinct 
value of R.c. Consider, for example, the following tu- 
ples for the relations R and S: 
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R.a 


R.b 


R.C 


S.C 


Car 


1 


Red 


Red 


Truck 


2 


Green 


Blue 


Boat 


4 


Blue 


Blue 


House 


0 


Orange 


Black 
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-continued 



R.a 


Rb 


R.c 


S.C 


House 


0 


Orange 





For every tuple of R having a value for R.c which does 
not join with a value of S.c for any tuple of S, a new 
tuple is generated by appending null values ( ) to that 
tuple of R. The tuples of R having a value for R.c which 
does not join with a value of S.c for any tuple of S will 
be referred to as the anti-joining tuples of R. Because a 
count over a specified attribute will not count null val- 
ues for that attribute, a count over the column S.c of the 
outer join will return a value of zero for the anti-join 
tuples: 



TEMPOJ (c, count) = 




R.c 


COLTNT (S.C) 


Red 


I 


Green 


0 


Blue 


2 


Orange 


0 
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Ganski's method avoids the count bug by providing a 
correct count associated with each distmct value of R.c. 
Ganski's method also provide proper un-nesting where 
the inner query does not have an equi-join correlation 
predicate. An equi-join correlation predicate is of the 
form fl(R)=f2(S) where fl and O are any functions on 
tuples of R and S, respectively. 

By applying Ganski's method to the nested query 
blocks of Example 2, the following un-nested queries 
are obtained: 



TEMP (name, count) = 

SELECT DEPT.name, COUNT (EMP.depi_name) 
FROM DEPT, EMP 

WHERE DEPT.name OJ EMP.dept_name 
40 GROUPBY DEPT.name 

SELECT DEPT.id 
FROM DEPT, TEMP 
WHERE DEPT.name = TEMP.name 

AND DEPT.pcs > TEMP.count 



A join with a join correlation predicate of R.c == S.c 
would create a new relation with the following tuples: 



R.a 


R.b 


R.C 


S.c 


Car 


1 


Red 


Red 


Boat 


4 


Blue 


Blue 


Boat 


4 


Blue 


Blue 



If the tuples were grouped by distinct joining values of 
R.c or S.c and aggregated by COUNT(S.c), the follow- 
ing relation TEMP would be generated: 



R.C 


S.C 


COUNT (S.c) 


Red 


Red 


1 


Blue 


Blue 


2 



The counts for this relation would be different from the 
counts for distinct values of R.c generated by iterative 
evaluation of the inner query in FIG. 9 because it would 
never return a count of zero. By replacing the join with 
a left outer join, a new relation with the following tu- 
ples would be generated: 



Ganski's method can convert two nested query 
blocks into a single query in SQL query language. In 
this case, the operation of the second query is per- 
formed by a "HAVING" statement. A GROUPBY 
statement specifies grouping according to a primary key 
or surrogate column having a distinct value for each 
tuple or row. Such an attribute will be designated by 
"#". The result for Example 2 is as follows: 

SELECT DEPT.id 

FROM DEPT, EMP 

WHERE DEPT.name OJ EMP.dept_name 
GROUPBY DEPT.# 

HAVING DEPT.pcs > COUNT (EMP.dept_jiame) 
In accordance with the present invention, there is 
provided an alternative method of un-nesting a nested 
query having a count aggregate. This alternative 
method is illustrated by the query graph in FIG. 12. The 
first query, represented by the node 198, is similar to the 
first query of Kim's method (node 193 in FIG. 10). In 
the second query, represented by node 197 of FIG. 12, 
the method of the present invention performs a (left) 
outer join, and applies a different predicate to the anti- 
joining tuples than to the joining tuples. In particular, 



R.a R.b R.c S.c 

65 

Car 1 Red Red 

Truck 2 Green 

Boat 4 Blue Blue 

Boat 4 Blue Blue 
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when a nested inner query has an equi-join predicate ski's method joins two base relations, whereas the 

joining a relation of the inner query to an outer query method of FIG. 12 joins a base relation with a tempo- 

and a count aggregate, the method of FIG. 12 removes rary relation. As a result, Ganski's method might be able 

the equi-join predicate from the inner query and places to employ more join methods, and the optimizer should 

a corresponding conjunctive (left) outerjoin predicate 5 take any alternative join methods into account. In any 

term in the predicate of the outer query, performs the event, the present invention provides a different way of 

count aggregate for each distinct value of the joining un-nesting a nested query having a COUNT aggregate 

attnbute of the relation of the inner query, and in the ^j^en the correlation predicates are all equi-joins. 

outer query apphes different predicates to the jomrng ^^^^ ^^ ^^^^ .^own a flowchart 

and anti-jommg tuples such that the predicate of the lo of a procedure for performing a left outer join and 

anti^mmng tuples is evaluated assmmng a count value ^p^j^f^g ^ ^^^^.^^^^ .^^ ^^^^ different from the 

° In Example I, for a count aggregate, the method of ^'^''^'^ ^PP''^'^ anti-jom tuples. In particular 

FIG. 12 gives the following t^o un-nested queries: the count associated with the anti-jom tuples is modified 

SO that it has a value of zero for the anti-jom tuples. The 
procedure of FIG. 13 takes advantage of the fact that 



TEMP (c, count) = the relation TEMP has a column with an attribute c that 

SELECT s,c, COUNT (•) jj^g distinct values. The row pointer P; to the relation 

FROM S . *' . 

GROUPBY s c TEMP is used as an index in an inner loop, and if no 



SELECT R.a matches to the distinct column c occur, then an anti-join 

FROMBUJ^MP^^^^ tuple is formed with the relation R, Otherwise, if a 

joining tuple is found, the inner loop is exited since the 



WHERE R.C OJ TEMP.C 

AND ( IF Rx =TEMP.c ... ^ , 

THEN R.b OPi TEMPxount J^m IS performed with a distmct value of TEMP.c, and 

ELSE R.b OPi 0) therefore no other joining tuple would be found by 

22 further iterations. 

As wiU be described below with respect to FIG. 13, ^ ^^^t step 221, a row pointer P/is allocated to the 

the IF-THEN-ELSE operations are best performed by relation TEMP, and a row pomter Vkis allocated to the 
dkectly applying the predicate operation OPi between relation R. Then, in step 222, the row pointers are set to 
the left join relation R and a count value when a joining zero. Both the inner and outer loops begm in step 223 
tuple of R is found, and applying the predicate opera- 30 with the join comparison between the row Pytand attri* 
tion OPi between the left join relation R and a value of bute c of the relation R and the row Pyand attribute c of 
0 when an anti-joining tuple of R is found. In the follow- the relation TEMP. If joining tuples are found, then the 
ing examples, the result of the IF-THEN-ELSE opera- inner loop is exited to step 224. In step 224, the predi- 
tions will be expressed simply as first the predicate cate for the joining tuples is applied. In this case, a test 
applied to the join tuples and second the predicate ap- 35 is made of the operation OPi between the attribute R.b 
plied to the anti-join tuples as follows: at row Pjt and the attribute TEMP.count at row Py, If 

[R.b OPi TEMP.count | R.b OPi 0]. With this nota- this predicate is found to be true in step 225, then in step 
tion, the query of Example 2 is un-nested as fol- 226, an output row is fonned by joining the Pjtth row of 
lows: the relation R with the Pjth row of the relation TEMP, 

4Q Returning now to step 223, if joining tuples are not 

found in step 223, then in step 227, the end of the inner 

SEl^CTEMP-^p^^t COUNT (•) ^^^p IS tested. If the row pomter P^is not equal to its 

FROM BMP maximum value PMXj, then the inner loop continues in 

GROUPBY EMP.dept—name Step 228 by incrementing the row pointer Py and execu- 

™^^?FJ!PtST^^^t, 45 tion loops back to step 223. If the row pointer P/reaches 

FROM DEPT, TEMP . . - . , .1 ■' . . . 

WHERE DEPT.name OJ TEMP dept_iianie maxunum value m Step 224, then there are no jouung 

AND [DEPT.pcs > TEMP.count | DEPT.pcs > 0] tuples for the value in the Pjtth row of the attribute R.c. 



Therefore, an anti-join tuple pair has been found. In this 
Under certain circumstances, the method of FIG. 12 ^ f ^"j^j" predicate is tested in step 229 that is 

may be more efficient than Ganski's method. The heu- 50 different from thejom predicate apphedm step 224. For 
ristic argument is based on (1) the number of tuples that anti-jom predicate 229, it is assumed that the value 

flow from each node in the query graphs corresponding ^f the attnbute TEMPxount has a value of zero. If this 
to the two methods, and (2) the number of tuples that antx-jom predicate is true, as tested m step 230, then m 
have to be processed by each groupby and outer join step 231, an output row is formed by joining the P^th 
operation. Both methods involve accessing relations R 55 row of the relation R with a value of the P^th row of the 
and S. Clearly [TEMPi | ^ lS|and |R| ^ |R OJ S|. attribute R.c, and the value of zero. In other words, the 
Assume that | S | < jR | . The number of tuples flowing row 231 is the row that would have been present m the 
from the groupby operation to the outerjoin operation absence of the so-called "count bug", 
in the method of FIG. 12 is equal to 1TEMP/| . The From steps 231 and 226, or if a false logical value is 
number of tuples flowing from ie outer join operation 60 found in steps 225 or 230, then in step 232, the end of the 
to the groupby operation in Ganski's method is equal to outer loop is tested by comparing the pointer Pjt to a 
I R OJ S I . Clearly | TEMPi | < | R OJ S | . The number predetermined maximum value PMXjt. If this maximiim 
of tuples processed by the groupby operation and the value is reached, execution returns. Otherwise, execu- 
outer join operation in the method of FIG. 12 is each tion continues in step 233 by setting the row pointer Py 
less than the corresponding number of tuples in Gan- 65 to zero, and incrementing the row pointer Pjt by one. 
ski's method. Hence, if | S | < 1 R [ ,the method of FIG. Execution then loops back to step 223. 
12 should perform better than Ganski's method. This The method of FIG. 4 is readily applicable to nested 
heuristic argument, however, ignores the fact that Gan- queries of more than two levels of nesting so long as the 
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correlation predicates of the COUNT blocks are 
"neighbor predicates". By definition, a "neighbor predi- 
cate" is an equi-join correlation predicate that refer- 
ences the relation in its own block and the relation from 
the immediately enclosing block. If a COUNT block 5 
does not have a "neighbor predicate", then the method 
of FIG. 12 is more difficult to apply than Ganski's 
method of FIG. 11, and consequently Ganski's method 
should be used for im-nesting of the count block. The 
use of the method of FIG. 12 for un-nesting blocks 10 
having more than two levels of nesting is shown in the 
following example: 
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EXAMPLE 4 



SELECT R.a 
FROMR 

WHERE R.b < (SELECT COUNT <*) 
FROMS 

WHERE R.c = S.c 
AND S.d > == (SELECT COUNT (*) 
PROMT 

WHERE S.e = T.e)) 



Notice that ail correlation predicates are neighbor pred- 
icates. 

Kim, suprUy page 465, disclosed that his method is 
appHcable to multiple nested blocks by applying it to 
pairs of inner and outer nested blocks beginning at the 
bottom of the hierarchical query graph. In other words, 
a search is made down to a leaf node of the query graph, 
and the method for un-nesting of a pair of blocks is 
apphed recursively to an inner nested leaf node and its 
outer parent node. Ganski, supra, pages 31-32, pro- 
posed a similar recursive -procedure upon the query 
graph to apply his method upon multiple nested blocks. 
The same approach can be used to apply the method of 
FIG. 12. The result of the query is obtained by evaluat- 
ing the following three queries. 
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TEMPI (e, count) = 
SELECT T.e. COUNT (♦) 
PROMT 
GROUPBY T.e 
TEMP2 (c, count) = 
SELECT S-c, COUNT (*) 
FROM S, TEMPI 
WHERE S.e OJ TEMPl.e 

AND [S.d > TEMPl.count ] S.d > = 0] 
GROUPBY S.C 
SELECT R.a 
FROM R, TEMP2 
WHERE R.C OJ TEMPZc 

AND [R-b < TEMP2.count ,' R.b < 0] 



Although recursive operations upon query graphs are 
both a convenient and readily understood way of ex- 55 
tending a method for \m-nesting two nested queries to 
nestings of multiple queries, an equivalent method 
would be to use symbolic execution upon algebraic 
expressions representing the relational operations of the 
nested queries. Nesting is symbolically represented in 60 
such algebraic expressions as a function which appears 
in an outer nested query and has an argument including 
the relational operations of an inner nested block. The 
method for evaluating the function for the pair of outer 
and inner nested blocks defines a rule or pattern which 65 
is applied or matched to the algebraic expression, and 
when a match is found, the substitution or change to the 
expression that should be performed to remove the 



function. Hardy^ supra, discloses further details regard- 
ing symbolic execution. 

The xm-nested query blocks themselves specify an 
order of evaluation that is not necessarily the best order 
due to the commutative and associative properties of 
joins, and similar properties apphcable to some opera- 
tions between joins and outer-joins. DayaU supra, ex- 
ploits these properties to generalize Ganski's solution 
for queries with more than 2 blocks. A linear query with 
multiple blocks gives rise to a 'linear J/OJ expression' 
where each instance of an operator is either a join or an 
outer join. A general linear J/OJ expression would like: 

R J/OJ S J/OJ T J/OJ U J/OJ . . . 

Relation R is associated with the outermost block, 
relation S with the next irmer block, and so on. An outer 
join is required if there is a COUNT between the re- 
spective blocks. In all other cases (AVG, MAX, MIN, 
SUM), only a join is required. The joins and outer joins 
are evaluated using the appropriate predicates. 

Since joins and outer joins do not commute with each 
other in general, a legal order may be obtained by com- 
puting all the joins first and then computing the outer 
joins in a left to right order (top to bottom, if you like) 
(Dayal, supra.) For example, the expression R OJ S J T 
J U OJ V J W can be legally evaluated as ((R OJ (S J T 
J U)) OJ (V J W)). Since the joins may be evaluated in 
any order, the least expensive join order is selected for 
joining relations S, T and U. 

Consider the following three block linear query: 
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SELECT Ra 
FROM R 

WHERE R.b OPi (SELECT COUNT (S.*) 
FROMS 

WHERE R.c OP2 S.c 
AND S,d OP3 (SELECT COUNT (T.*) 
PROMT 

WHERE S.e OP4 Xe 
AND R.f OP5 T.Q) 

The corresponding linear expression is R OJ S OJ T 
and hence a legal order is (R OJ S) OJ T. The result is 
obtained by executing the following two queries: 
Query 1: 

TEMPi (#, a, b, *)- 

SELECT R.#, R-a, R.b, S.* 

FROM R, S, T 

WHERE OJ S) OJ T 

GROUP BY R.#, S.# 

HAVING S.d OP3 COUNT(T.*) 
Query 2 

SELECT TEMPi.a 

FROM TEMPi 

GROUP BY TEMPi.# 

HAVING TEMPi.b OPiCOUNT(TEMPi.*) 

The outer join predicates are implicit in Query 1. The 
predicate for R OJ S is (R.c OP2 S.c), while the predi- 
cate for the second outer join with T is (S.e OP4T.e and 
R.f OP5 T.f)- Tuples from Query 1 may be pipelined 
into Query 2. Notice that if the query has d blocks, the 
total mmiber of joins and outer joins will be (d-1). These 
will be followed by (d-1) groupby-having operations. 

Although a valid J/OJ ordering is obtained by per- 
forming all the joins first, followed by the outer joins 
from left to right, it is sometimes possible to change this 
order. Consider, for example, the following query: 
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SELECT R.a 
FROM R 

WHERE R-b OPi (SELECT COUNT (S.*) 

FROMS 5 

AND S.d OP2 (SELECT MAX (T.d) 
PROMT 
WHERE R.fOP3 T.{)) 



The J/OJ expression for the above query is R OJ (S J 10 
T). Since there is no correlation predicate between the 
S and T relations, a cartesian product must be per- 
formed to compute (S J T). The outer join is then per- 
formed using the predicate (R.f OP3 T.f). However, for 
each (r, s) pair, where r e R and s e S, MAX (T.d) de- 15 
pends only on r. Hence, we can precompute MAX (T.d) 
associated with each tuple of R as follows: 

TEMPi (#, a, b, max)= 

SELECT R.#, R-a, R-b, MAX (T.d) 

FROM R, T 20 

WHERE R.f OP3 T.f— OJ 

GROUP BY R. # 

Notice that | TEMPi | = | R | . Essentially, TEMPj has 
all the attributes of R required for further processing 
along with the MAX (T.d) associated with each tuple of 25 
R. MAX (T.d) can be computed in this fashion because 
it occurred in the last block. Any aggregate that does 
not occur in the last block depends on the results of the 
blocks below it and hence cannot be evaluated before 
the blocks below it are evaluated. Also, an outer join 39 
was performed between R and T even though we were 
computing MAX (T.d). This is because COUNT (S.*) 
indirectly depends on each tuple of R as R is referenced 
inside the third block which is nested within the second 
block. Hence, all tuples of R must be preserved. For a 35 
tuple of R with no joining tuples in T, the MAX value 
is set to a null value ( * ). (Any comparison where one 
or both of the operands is null ( * ) evaluates to un- 
known, which SQL regards as false for query evalua- 
tion purposes.) The original query is now re-written as 40 
follows: 



SELECT TEMPi-a 
FROM TEMPi 

WHERE TEMPj.b OPj (SELECT COUNT (S.*) 45 
FROMS 

WHERE S.d OP2 TEMP 1 .max) 



We now have a correlation predicate between 
TEMPi and S, thus avoiding a cartesian product. Simi- 50 
lar ideas were presented in Dayal, supra, in his section 
titled "Positioning G-Agg Operations". In that section, 
Dayal presents rules for computing aggregates before 
G-joins. 

In any case, the above example shows that it is possi- 55 
ble to precompute the bottom-most aggregate (BMA) if 
the number of outer relations referenced in the last 
block have already been joined. Although in this exam- 
ple the bottom-most aggregate depended only on one 
outer relation, a further example is presented below 60 
where the bottom-most aggregate depends on more 
than one relation. 

It is also possible to evaluate pipelined queries in a 
strictly top-down order, perfonning the joins and outer 
joins in the order they occur. Evaluation in a top-down 65 
manner may permit the method of FIG. 12 to be applied 
to a larger number of contiguous blocks at the end of 
the query. However, care must be taken to ensure that 
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any join that is present just below an outer join is also 
evaluated as an outer join. Consider the following ex- 
ample: 



SELECT R.a 
FROMR 

WHERE R.b OPi (SELECT COUMT (S .*) 
FROMS 

WHERE R-c OP2 S.C 
AND S.d OP3 (SELECT MAX (T.d) 
PROMT 

WHERE S.e OP4 T.e 
AND R.fOP5T.O) 



The J/OJ expression is R OJ (S J T) . The join predi- 
cate between S and T is (S.e OP4T.e) and the outer join 
predicate is (R.c OP2 S.c and R.f OPsT.f). Assume that 
the join between S and T is very expensive and should 
be possibly avoided. Could we evaluate (R OJ S) first? 
It turns out that we can indeed perform (R OJ S) first. 
However, some precautions/modifications are neces- 
sary. 

It is clear that if an R tuple has no matching S tuples, 
the count associated with that R tuple is 0. As pointed 
out in Muraliy supra, this R tuple may be routed to a 
higher node in the query tree so that it does not partici- 
pate in the next join operation with T, 

We thiis need to consider only the join tuples of the 
form (r, s) from the outer join, where r € R and s € S. Let 
us focus our attention on a single tuple of R. When the 
join with T is evaluated using the predicate (S.e OP4T.e 
AND R.f OPsT.f), it is quite possible that none of these 
(r, s) tuples join with any tuples of T. In this case, the r 
tuple will be lost. However, if r.b equals 0, r is a result 
tuple and hence must be preserved. On the other hand, 
if some of the (r, s) tuples do join with some T tuples, it 
may so happen that after we do the groupby by (R. #, 
S. #) and evaluate MAX (T.d), none of the s.d values in 
the (r, s) tuples satisfy (s.d OP3MAX (T.d)). We may be 
tempted to discard all the (r, s) groups. Again, if r.b 
equals 0, we need to preserve r. 

We can preserve r if we perform the next join as an 
outer join. Also, the groupby operator must not discard 
any (r, s) group not satisfying (s.d OP3MAX (T.d)). 
Instead, it must pass it on preserving the R portion of 
the tuple and nulling out the S portion of the tuple. 

The same ideas were presented in Murali, supra, 
when un-nesting tree queries. Summarizing, if we en- 
counter the expression R OJ S OJ T J U J V, we could 
evaluate it as ((R OJ S) OJ (T J U J V)). The above 
order corresponds to evaluating ail the joins first. An- 
other evaluation order could be ((((R OJ S) OJ T) OJ 
(U J V)). Now we have an outer join between T and U. 
Carrying this idea one step further, the above expres- 
sion may also be evaluated as ((((R OJ S) OJ T) OJ U) 
OJ V). 

By applying the methods of FIGS. 10, 11 and 12 to 
pairs of blocks in a nested query having multiple blocks, 
it is possible to generate a multiplicity of alternative 
query graphs. The query optimizer may compute a cost 
associated with each query graph, and select the least 
expensive graph for execution. 

In the following example, it will be assumed that the 
query optimizer operates on a kind of "join graph". An 
example of such a join graph is shown in FIG. 14. The 
graph G = (V, E) consists of a set of vertices V and a set 
of directed edges E. There is a one-one correspondence 
between the blocks of the query and the elements of V. 
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Each element of V, except for the first vertex, is labeled 
either C (COUNT) or NC (Non COXJNT). This label- 
ing is clearly suggestive of the kind of aggregate 
(COUNT or Non COUNT) present in that block. The 
vertices are numbered 1 through d, where d is the cur- 5 
rent number of vertices in the graph. A directed edge is 
drawn from vertex i to j (i < j) if there is a correlation 
predicate in the jth block between the relations of 
blocks i and j. 

The method of FIG. 12 may be apphed to the last k 10 
blocks of a query (O^k^d) if the last k vertices of the 
graph of the query satisfy the following properties: 

(1) The in degree of every C vertex is at most 1; 

(2) The edge incident on a C vertex corresponds to a 
neighbor predicate; 15 

(3) All the edges incident with the last k vertices 
correspond to equi-join correlation predicates; and 

(4) The relations in the first d-k blocks have already 
been joined. 

The bottom-most aggregate may be precomputed if 20 
the in degree of the last vertex is at most 1. 
The operations on the graph are as follows: 

(1) When the relations of two or more blocks are 
joined, the corresponding vertices are collapsed 
into one vertex. The edges adjacent to these verti- 25 
ces are removed, while all the edges that connect 
these vertices to other vertices are preserved. Mul- 
tiple edges are replaced by a single edge. 

(2) Let d-1 and d be the last two vertices in the graph. 

If the BMA is computed, the last vertex d is re- 30 
moved from the graph and the edge incident on d 
is connected to d-1. 
Notice that we may be able to apply the method of 
FIG. 12 only after joining some relations. For example, 
we may apply the method of FIG. 12 to the last block 35 
after joining R and S in the query of Example 3. This is 
because the predicate (RS—T.f) becomes a neighbor 
predicate only after relations R and S are joined. Thus, 
the number of blocks for which we may apply the 
method of FIG. 12 can change dynamically. Similarly, 40 
the bottom-most aggregate may have originally de- 
pended on more than one outer relation, but, after these 
relations have been joined, the in degree of the last 
vertex will become 1. The bottom-most aggregate may 
be precomputed at this point. . 45 

When a series of consecutive m joins are encountered 
in a J/OJ expression, one may be tempted to evaluate all 
the joins using the cheapest order. However, the joins 
should be evaluated incrementally. In other words, the 
first i joins should be evaluated at a time, where 50 
1 =i^m. This ensures that we may be able to apply the 
method of FIG. 12 to a larger group of contiguous 
blocks at the end of the query. 

Turning now to FIG. 15, there is shown a flowchart 
of an un-nesting procedure that incorporates the above 55 
considerations for un-nesting multiple nested quires. In 
the first step 251 of FIG. 15, execution branches de- 
pending on whether the bottom-most aggregate can be 
precomputed. It can be precomputed if the predicate of 
its node does not reference a relation in the relations list 60 
of any other nodes in which the bottom-most aggregate 
is nested. If so, then in step 252, the bottom-most aggre- 
gate is computed and its corresponding node is removed 
from the query graph. Then, in step 253, the query 
procedure of FIG. 15 is called recursively to process 65 
the amended query graph (G')- Execution then returns. 
When step 251 finds that the bottom-most aggregate 
cannot be precomputed, then in step 254, the nodes are 



identified to which Kim's method is appHcable. These 
are the nodes that have aggregates other than COUNT, 
and which have equi-join predicates, and nodes with 
equi-join predicates and COUNTs known to always 
generate a count >0. Then, in step 255, execution 
branches when Kim's method can be applied to all 
nodes. If so, in step 256, Kim's method is applied from 
the bottom to the top of the query graph, and execution 
returns. 

If Kim*s method cannot be applied to all nodes, then 
in step 257, the aggregate nodes to Which Kim's 
method cannot be applied are inspected to determine 
whether the method of FIG. 12 is applicable. The 
method of FIG. 12 is applicable to the count nodes with 
equi-join predicates that are neighbor predicates. If so, 
then in step 258, the method of FIG. 12 or Ganski's 
method is chosen for these nodes depending upon a 
comparison of the computational cost of each respec- 
tive method to each particular node. In step 259, Gan- 
ski's method is chosen for the remaining aggregate 
nodes, because it is the only practical method. Then, in 
step 260, possible execution orders are considered, be- 
ginning first with the possibility of joins followed by 
outer-joins. If joins are followed by outer-joins, then in 
step 261, for i— 1 to m, the first i joins are evaluated 
using the least costly join order. Then, in step 262, the 
un-nesting procedure of FIGS. 15 and 16 is recursively 
called for the new graph including the remaining nodes, 
and execution returns. 

If step 260 determines that there is not a sequence of 
joins followed by outer-joins, then execution branches 
to step 263. In step 263, execution returns if there is not 
a sequence of joins within outer joins. If so, then in step 
264, for i = 1 to m, the fu-st i joins are evaluated using the 
least cosdy join order. Then, in step 265, the un-nesting 
procedure of FIGS. 15 and 16 is called recursively to 
operate upon the graph (G') of the remaining nodes. 
Then, in step 266, the first outer join is evaluated, and 
the first join is replaced by an outer join. Then, in step 
267, the un-nesting procedure of FIGS. 15 and 16 is 
called recursively with the graph (G') including the 
remaining nodes. Execution then returns. 

Consider the application of the procedure of FIGS. 
15 and 16 to the following nested query, represented by 
the join graph of FIG. 14: 



SELECT R.a 
FROMR 
WHERE R.b OPi 

(SELECT COUNT (S.*) 

FROMS 

WHERE R.C = S.c 
AND S.d OP2 

(SELECT AVG (T.d) 

PROMT 

WHERE S.e T.e 
AND T.g OP3 

(SELECT SUM (U.g) 

FROMU 

WHERE S,h = U-h 
AND T.i = U.i))) 

The J/OJ expression is R - - - OJ - - - S - - - J - - - T 

J XJ. The alternative query plans shown in 

FIGS. 17 to 22 are possible. 

In FIG. 17, Kim's method and the method of FIG. 12 
are applied to blocks 2, 3 and 4. 

In FIG. 18, R and S are joined by Ganski's method, 
and Kim's method is applied to blocks 3 and 4, Since the 
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outer join between R and S is performed before the join, first and second relations and alphanumeric symbols for 

the first join is now evaluated as an outer join. operators in said first and second predicates. 

In FIG. 19, R, S and T are joined by Ganski's 3. The method as claimed in claim 2, wherein said 

method, and Kim's method is applied to block 4. Notice inner query includes a select statement including the 

that both joins are now replaced by outer joins. 5 words "SELECT" and "COUNT" and said first predi- 

In FIG. 20, Ganski's method is used to replace all cate is included in a where statement including the 

joins by outer joins, followed by three groupby opera- word "WHERE". 

tions. Ths method as claimed in claim 2, further compris- 

In FIG. 21, Kim's method is used to join relations S, ing the step of parsing said inner and outer query to 

T and U first, followed by outer joins from Ganski's 10 form a linked data structure mcluding a first node repre- 

method. senting said iamer query and a second node representing 

In FIG. 22, Kim's method is used to join relations S, said outer query. 

T and U first. Since the bottom-most aggregate depends ^- ^h^ method as claimed in claim 4, wherein said 

only on relations S and T, the bottom-most aggregate is steps of convertmg generate another linked data struc- 

computed before the outer join with R. ture including a third node defming said first un-nested 

Notice that it was important to evaluate the joins ^^^^ ^ ^^^^ defming said second un-nested 

incrementally. Most of the outer join nodes in FIGS. 17 ^^t^' , , . . , . . . 

to 22 have two output edges. The vertical edge repre- 6. The method as clauned m claun 5, wherem said 

sents the anti-jom tuples, while the other edge repre- ^f^^' ^^^^ structure mcludes a senes of nodes 

sents the jom tuples. Similarly, the groupby-having ^^^"^S ^ sequence of jom and outer-jom operations 

nodes have two output edges. TTie vertical edge repre- mciudmg a consecutive senes of m joms, and wherem 

sents the groups that did not satisfy that condition in the ^he furst i joms are evaluated at a tune fust for i^ 1, then 

having clause These groups have certain portions for then for i==3 . . and fmally for 

,1 j° TT 1 • r^rr^ r^ ' The mcthod as Claimed m claim 6, wherem said 

nulled out. For example, m FIG, 20, groups flowmg „^ . . ^ ^- i- i j j ^ x . 

- xi_ r- X ^ L ' ji r ^ * 25 evaluatmg generates a respective Imked data structure 

from the first groupby-havmg node to the topmost - i. i r • 4.- 

. ^ . . i ^- , J r^t- for each value of i representmg an alternative query 

groupby-havmg node along the vertical edge are of the , & j 

form (R, NULL). Also, FIGS. 18 to 22 have edges that ^ f '^^ ^^^^^^ ^ ^^^^^ ^ ^^^^^^ 

route tuples to a node much higher m the tree than the j^j^^^ ^^^^^^^^ i^erarchical and includes a mul- 

unmediate parent As pomted out m Murali, supra, this 3^ ^ ^^^^^^^ descendant from a root node, and said 

IS optional but leads to savings m message costs. converting are performed upon parent-child 

In view of the above, there, has been descnbed a ^^^^^ ^^^^ ^^il^ ^^^^^ 1^^^ ^^^^^ 

relational database system mciudmg a query optmiizer linked data structure. 

that has available to it an alternative method for un- 9. The method as claimed in claun 1, further compris- 

nestmg queries that mclude a COUNT aggregate. In 35 ing the step of executing said second un-nested query by 

many cases the alternative method provides a more indexing said results of said furst un-nested query with a 

efficient way of executmg the un-nested queries. In first index and mdexing values of said furst relation with 

other cases the alternative method permits alternative ^ second mdex, and for each value of said first relation 

query plans to be generated which may result in a sav- mdexed by said second index, sequencing said 

ings in the total cost of the query. 40 f^j.^^ in^iex and iteratively testing said first predicate 

What IS claimed is: ^j^h said each value of said first relation and one of said 

1. A method of operating a digital computer for un- results mdexed by said Gist index, and when said testing 
nestuig an inner query from an outer query, said inner fi^ds that said fu^t predicate is true, testmg said second 
query referencmg a furst relation also referenced in said predicate with said each value of said furst relation and 
outer query, said inner query including a first predicate 45 the indexed one of said results and terminating said 
joining said first relation to a second relation, said inner testing of said first predicate with said each value of said 
query also including a count aggregate, said outer query first relation, and when said testing has tested said fu-st 
havmg a second predicate referencing said furst relation predicate for said each value of said furst relation and all 
and said inner query, said method comprising the steps of said results without ever finding that said first predi- 
on: 50 cate is true, applying said second predicate to said each 

a) convertmg said inner query to a fu^t un-nested value of said fu-st relation and a value of zero. 

query by removing said fu-st predicate and modify- 10. A method of operating a digital computer for 

ing said count aggregate function to count over executing an outer query including a nested inner query, 

groups of distinct values of said second relation; said inner query referencing a first relation also refer- 

and 55 enced in said outer query, said inner query including a 

b) converting said outer query to a second un-nested first predicate joining said furst relation to a second 
query receivmg results of said inner query by modi- relation, said inner query also including a count aggre- 
fying said second predicate so that said second gate, said outer query having a second predicate refer- 
predicate is applied to said first relation and said encing said first relation and said inner query, said 
results for values of said first relation which are 60 method comprising the steps of: 

joined to said resists by said fu-st predicate and so a) computing a third relation including values of said 

that said second predicate is applied to said first second relation by counting over groups of distinct 

relation and a value of zero for values of said first values of said second relation to compute a count 

relation which are not joined to any of said results for each distinct value of said second relation; and 

by said frrst predicate. 65 b) applying said first predicate to combinations of 

2. The method as claimed m claim 1, wherein said values of said first relation and the distinct values of 
inner query and said outer query are expressed in a said second relation, and for combinations of values 
query language including alphanumeric names for said of said first relation and distinct values of said sec- 



25 



5,412,804 



26 



ond relation which satisfy said first predicate, ap- 
plying said second predicate to said first relation 
and the count of said third relation for each distinct 
value of said second relation, and for values of said 
first relation for which said first predicate is not 5 
satisfied in combination with any of said distinct 
values of said second relation, applying said second 
predicate to said first relation and a value of zero, 

11. The method as claimed in claim 10, wherein said 
inner query and said outer query are expressed in a 10 
query language including alphanumeric names for said 
first and second relations and alphanumeric symbols for 
operators in said first and second predicates. 

12. The method as claimed in claim 11, wherein said 
inner query includes a select statement including the 15 
words "SELECT" and "COUNT", and said first predi- 
cate is included in a where statement including the 
word "WHERE". 

13. The method as claimed in claim 11, further com- 
prising the step of parsing said inner and outer query to 20 
form a linked data structure including a first node repre- 
senting said inner query and a second node representing 
said outer query. 

14. The method as claimed in claim 13, further com- 
prising the step of converting said linked data structure 25 
into another linked data structure including a third node 
defining a first un-nested query for generating said third 
relation and a fourth node defining a second un-nested 
query for receiving said third relation, and wherein said 
step (a) of computing is performed by executing said 30 
first im-nested query and said step (b) of applying is 
performed by executing said second un-nested query. 

15. The method as claimed in claim 14, wherein said 
another linked data structure includes a series of nodes 
defining a sequence of join and outer-join operations 35 
including a consecutive series of m joins, and wherein 
the first i joins are evaluated at a time first for i= 1, then 
for i=2, then for i=3, . . . , and finally for i=m. 

16. The method as claimed in claim 14, wherein said 
linked data structure formed by said parsing is hierar- 40 
chical and includes a multiplicity of nodes descendant 
from a root node, and said step of converting is per- 
formed upon parent-child pairs of nodes when the child 
nodes are leaf nodes in said linked data structure formed 
by said parsing. 45 

17. The method as claimed in claim 10, wherein said 
step (b) of applying includes indexing said third relation 
with a first index and indexing all values of said first 
relation with a second index, and for each value of said 
first relation, sequencing said first index and iteratively 50 
testing said first predicate with said each value of said 
first relation and one of said values of said second rela- 
tion indexed by said first index, and when said testing 
finds that said first predicate is true, testing said second 
predicate with said each value of said first relation and 55 
the count for the indexed one of said values of said 
second relation and terminating said testing of said first 
predicate with said each value of said first relation, and 
when said testing has tested said first predicate for said 
each value of said first relation and all of said values of 60 
said second relation from said third relation without 
ever finding that said first predicate is true, applying 
said second predicate to said each value of said first 
relation and a value of zero. 

18. A database system comprising, in combination: 65 
a memory having stored in it a relational database; 

an input device for receiving a query from a user 
requesting data from said relational database; 



an output device for transmitting to the user data 
from said relational database; and 

a query processing system including means for pars- 
ing said query to generate a query graph in said 
memory representing said query, means for gener- 
ating an optimized query from said query graph; 
and means for executing said optimized query; 

wherein said means for generating an optimized 
query includes means for un-nesting an inner query 
from an outer query, said inner query referencing a 
first relation also referenced in said outer query, 
said inner query including a first predicate joining 
said first relation to a second relation, said inner 
query also including a count aggregate, said outer 
query having a second predicate referencing said 
first relation and said inner query, said means for 
generating including: 

a) means for converting said inner query to a first 
un-nested query by removing said first predicate 
and modifying said count aggregate function to 
count over groups of distinct values of said second 
relation; and 

b) means for converting said outer query to a second 
un-nested query receiving results of said inner 
query by modifying said second predicate so that 
said second predicate is applied to said first relation 
and said results for values of said first relation 
which are joined to said results by said first predi- 
cate and so that said second predicate is applied to 
said first relation and a value of zero for values of 
said first relation which are not joined to any of 
said results by said first predicate. 

19. The relational database system as claimed in claim 
18, wherein said means for executing, said optimized 
query includes means for executing said second un- 
nested query by indexing said results of said first un- 
nested query with a first index and indexing all values of 
said first relation with a second index, and for each 
value of said first relation that is indexed by said second 
index, sequencing said first index and iteratively testing 
said first predicate with said each value of said first 
relation and one of said results indexed by said first 
index, and when said testing finds that said first predi- 
cate is true, testing said second predicate with said each 
value of said first relation and the indexed one of said 
results and terminating said testing of said first predicate 
with said each value of said first relation, and when said 
testing has tested said first predicate for said each value 
of said first relation and all of said results without ever 
finding that said first predicate is true, applying said 
second predicate to said each value of said first relation 
and a value of zero. 

20. The method as claimed in claim 18, wherein said 
means for parsing includes means for parsing a series of 
a multiplicity of nested queries to generate a query 
graph including a multiplicity of linked nodes, said 
means for un-nesting includes means for generating 
another query graph including a series of a multiplicity 
of linked nodes representing un-nested queries each 
having either a join or an outer-join operation and to- 
gether defining a sequence of join and outer-join opera- 
tions including a consecutive series of m joins, and 
wherein said means for optimizing further includes 
means for generating a plurality of alternative query 
graphs by evaluating the first i joins at a time first for 
i=l, then for i=2, then for i— 3, . . . , and finally for 
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Appendix C; Related Proceedings 

None. Appellant, the undersigned Attorney for Appellant, and the Assignee know of 
no applications on appeal that may directly affect or be directly affected by or have a bearing 
on the Board's decision in the pending appeal. 
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